2024-04-26T01:32:35.997 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T01:32:35.997 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} --owner scheduled_leonidus@teuthology --name leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi --archive /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259 -- /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/orig.config.yaml 2024-04-26T01:32:36.056 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T01:32:36.145 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} email: null first_in_suite: false job_id: '7674259' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: afa1933b0fdbb6c99c947d1eda34d661d23cd327 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3448 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: afa1933b0fdbb6c99c947d1eda34d661d23cd327 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGizPNmw1LlSGwIrePUHjicINJICEBxG5sdGO/VeJgpDkeL5MDzOgLLVxmKRIEIlHoElNXfCK4PjfFJFhdQiowI= smithi045.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLQ/TmU7YkKUqL56CIZ+ETMcUmuyvEFvQKsrcROCAIBzyoDbPBgK7CRpw1qGYYYVx9feOS+WBubI4L5ZAlNC40Q= smithi071.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8YniUF/fg0t96N31fjnnvQR6W/pwoKLRwec+qW8tWymJWYHRGNmzVYE/JmOZ/mHmrYMkOjP0WJJq1WxFyD+hY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_01:02:28 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T01:32:36.145 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa; will attempt to use it 2024-04-26T01:32:36.146 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_afa1933b0fdbb6c99c947d1eda34d661d23cd327/qa/tasks 2024-04-26T01:32:36.146 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T01:32:36.148 INFO:teuthology.task.internal:Checking packages... 2024-04-26T01:32:36.171 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-26T01:32:36.171 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T01:32:36.171 INFO:teuthology.packaging:ref: None 2024-04-26T01:32:36.171 INFO:teuthology.packaging:tag: None 2024-04-26T01:32:36.171 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-26T01:32:36.171 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:32:36.172 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-26T01:32:36.550 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-26T01:32:36.551 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T01:32:36.561 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T01:32:36.561 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T01:32:36.574 INFO:teuthology.task.internal:Saving configuration 2024-04-26T01:32:36.596 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T01:32:36.611 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T01:32:36.635 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:27:27.537549', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGizPNmw1LlSGwIrePUHjicINJICEBxG5sdGO/VeJgpDkeL5MDzOgLLVxmKRIEIlHoElNXfCK4PjfFJFhdQiowI='} 2024-04-26T01:32:36.655 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi045.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:27:27.536184', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLQ/TmU7YkKUqL56CIZ+ETMcUmuyvEFvQKsrcROCAIBzyoDbPBgK7CRpw1qGYYYVx9feOS+WBubI4L5ZAlNC40Q='} 2024-04-26T01:32:36.674 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi071.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 01:27:27.534153', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBN8YniUF/fg0t96N31fjnnvQR6W/pwoKLRwec+qW8tWymJWYHRGNmzVYE/JmOZ/mHmrYMkOjP0WJJq1WxFyD+hY='} 2024-04-26T01:32:36.674 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T01:32:36.685 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T01:32:36.685 INFO:teuthology.task.internal:roles: ubuntu@smithi045.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T01:32:36.685 INFO:teuthology.task.internal:roles: ubuntu@smithi071.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T01:32:36.685 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T01:32:36.800 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f150035a0d0>, signals=[15]) 2024-04-26T01:32:36.800 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T01:32:36.809 INFO:teuthology.task.internal:Opening connections... 2024-04-26T01:32:36.809 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T01:32:36.811 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:32:36.891 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi045.front.sepia.ceph.com 2024-04-26T01:32:36.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:32:36.965 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi071.front.sepia.ceph.com 2024-04-26T01:32:36.966 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:32:37.039 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T01:32:37.046 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-04-26T01:32:37.066 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-04-26T01:32:37.066 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:32:37.124 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-04-26T01:32:37.125 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:32:37.125 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:32:37.125 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:32:37.125 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:32:37.125 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:32:37.125 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:32:37.126 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-04-26T01:32:37.152 DEBUG:teuthology.orchestra.run.smithi045:> uname -m 2024-04-26T01:32:37.171 INFO:teuthology.orchestra.run.smithi045.stdout:x86_64 2024-04-26T01:32:37.171 DEBUG:teuthology.orchestra.run.smithi045:> cat /etc/os-release 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:NAME="CentOS Stream" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION="9" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ID="centos" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:VERSION_ID="9" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:ANSI_COLOR="0;31" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:32:37.229 INFO:teuthology.orchestra.run.smithi045.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:32:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:32:37.230 INFO:teuthology.orchestra.run.smithi045.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:32:37.230 INFO:teuthology.lock.ops:Updating smithi045.front.sepia.ceph.com on lock server 2024-04-26T01:32:37.252 DEBUG:teuthology.orchestra.run.smithi071:> uname -m 2024-04-26T01:32:37.269 INFO:teuthology.orchestra.run.smithi071.stdout:x86_64 2024-04-26T01:32:37.270 DEBUG:teuthology.orchestra.run.smithi071:> cat /etc/os-release 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:NAME="CentOS Stream" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION="9" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:ID="centos" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:ID_LIKE="rhel fedora" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:VERSION_ID="9" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:PLATFORM_ID="platform:el9" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:ANSI_COLOR="0;31" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:LOGO="fedora-logo-icon" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:HOME_URL="https://centos.org/" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T01:32:37.328 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T01:32:37.329 INFO:teuthology.orchestra.run.smithi071.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T01:32:37.329 INFO:teuthology.lock.ops:Updating smithi071.front.sepia.ceph.com on lock server 2024-04-26T01:32:37.360 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T01:32:37.371 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T01:32:37.380 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T01:32:37.380 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:32:37.383 DEBUG:teuthology.orchestra.run.smithi045:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:32:37.385 DEBUG:teuthology.orchestra.run.smithi071:> test '!' -e /home/ubuntu/cephtest 2024-04-26T01:32:37.401 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T01:32:37.409 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T01:32:37.410 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:32:37.441 DEBUG:teuthology.orchestra.run.smithi045:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:32:37.443 DEBUG:teuthology.orchestra.run.smithi071:> test -z $(ls -A /var/lib/ceph) 2024-04-26T01:32:37.472 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T01:32:37.554 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T01:32:37.570 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-26T01:32:37.570 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}} 2024-04-26T01:32:37.570 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}}, timeout 300 2024-04-26T01:32:37.571 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:32:37.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:32:37.589 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-26T01:32:37.919 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:32:37.919 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-26T01:32:37.919 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-26T01:32:37.937 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-437.el9.x86_64 2024-04-26T01:32:37.937 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:32:37.938 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-26T01:32:37.938 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum upgrade -y linux-firmware 2024-04-26T01:32:40.524 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 7.9 MB/s | 8.0 MB 00:01 2024-04-26T01:32:42.391 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-26T01:32:46.842 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-04-26T01:32:48.282 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 16 kB 00:00 2024-04-26T01:32:51.479 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 21 MB 00:03 2024-04-26T01:32:56.692 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-04-26T01:32:58.713 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T01:32:58.714 INFO:teuthology.orchestra.run.smithi037.stdout:Nothing to do. 2024-04-26T01:32:58.714 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T01:32:58.812 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T01:32:58.812 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-26T01:32:59.050 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-26T01:32:59.051 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-26T01:32:59.053 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi037.front.sepia.ceph.com to complete... 2024-04-26T01:33:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26 01:33:12 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-26T01:33:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:FINISHED --2024-04-26 01:33:12-- 2024-04-26T01:33:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:Total wall clock time: 14s 2024-04-26T01:33:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:Downloaded: 1 files, 341M in 13s (25.3 MB/s) 2024-04-26T01:33:12.887 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-26T01:33:12.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-26T01:33:15.080 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying... ######################################## 2024-04-26T01:33:15.238 INFO:teuthology.orchestra.run.smithi037.stdout:Preparing... ######################################## 2024-04-26T01:33:15.356 INFO:teuthology.orchestra.run.smithi037.stdout:Updating / installing... 2024-04-26T01:33:23.680 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-26T01:34:03.482 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-26T01:34:03.482 DEBUG:teuthology.orchestra.run.smithi037:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-26T01:34:03.582 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-26T01:34:03.583 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-26T01:34:03.584 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-26T01:34:03.585 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-26T01:34:03.586 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-26T01:34:03.587 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-26T01:34:03.588 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-26T01:34:03.589 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-26T01:34:03.590 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-26T01:34:03.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-26T01:34:03.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-26T01:34:03.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-26T01:34:03.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-26T01:34:03.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-26T01:34:03.591 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-26T01:34:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-26T01:34:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-26T01:34:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-26T01:34:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-26T01:34:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-26T01:34:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-26T01:34:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-26T01:34:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-26T01:34:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-26T01:34:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-26T01:34:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-26T01:34:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-26T01:34:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-26T01:34:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-26T01:34:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-26T01:34:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-26T01:34:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-26T01:34:03.599 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-26T01:34:03.600 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-26T01:34:03.601 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-26T01:34:03.602 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-26T01:34:03.603 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-26T01:34:03.604 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-26T01:34:03.605 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-26T01:34:03.606 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-26T01:34:03.607 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-26T01:34:03.608 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-26T01:34:03.609 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-26T01:34:03.610 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-26T01:34:03.611 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-26T01:34:03.612 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-26T01:34:03.613 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-26T01:34:03.614 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-26T01:34:03.615 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-26T01:34:03.616 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-26T01:34:03.617 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-26T01:34:03.618 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-26T01:34:03.619 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-26T01:34:03.620 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-26T01:34:03.621 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-26T01:34:03.622 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-26T01:34:03.623 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-26T01:34:03.624 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-26T01:34:03.625 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-26T01:34:03.626 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-26T01:34:03.627 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-26T01:34:03.628 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-26T01:34:03.629 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-26T01:34:03.630 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-26T01:34:03.631 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-26T01:34:03.632 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-26T01:34:03.633 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-26T01:34:03.634 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-26T01:34:03.635 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-26T01:34:03.636 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-26T01:34:03.637 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-26T01:34:03.638 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-26T01:34:03.639 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-26T01:34:03.640 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-26T01:34:03.641 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-26T01:34:03.642 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-26T01:34:03.643 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-26T01:34:03.644 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-26T01:34:03.645 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-26T01:34:03.646 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-26T01:34:03.647 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-26T01:34:03.648 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-26T01:34:03.649 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-26T01:34:03.650 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-26T01:34:03.651 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-26T01:34:03.652 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-26T01:34:03.653 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-26T01:34:03.654 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-26T01:34:03.655 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-26T01:34:03.656 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-26T01:34:03.657 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-26T01:34:03.658 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-26T01:34:03.659 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-26T01:34:03.660 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-26T01:34:03.661 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-26T01:34:03.662 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-26T01:34:03.663 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-26T01:34:03.664 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-26T01:34:03.665 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-26T01:34:03.666 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-26T01:34:03.667 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-26T01:34:03.668 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-26T01:34:03.669 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-26T01:34:03.670 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-26T01:34:03.671 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-26T01:34:03.672 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-26T01:34:03.673 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-26T01:34:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-26T01:34:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-26T01:34:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-26T01:34:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-26T01:34:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-26T01:34:03.674 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-26T01:34:03.675 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-26T01:34:03.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-26T01:34:03.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-26T01:34:03.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-26T01:34:03.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-26T01:34:03.676 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-26T01:34:03.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-26T01:34:03.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-26T01:34:03.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-26T01:34:03.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-26T01:34:03.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-26T01:34:03.677 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-26T01:34:03.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-26T01:34:03.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-26T01:34:03.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-26T01:34:03.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-26T01:34:03.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-26T01:34:03.678 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-26T01:34:03.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-26T01:34:03.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-26T01:34:03.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-26T01:34:03.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-26T01:34:03.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-26T01:34:03.679 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-26T01:34:03.680 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-26T01:34:03.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-26T01:34:03.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-26T01:34:03.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-26T01:34:03.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-26T01:34:03.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-26T01:34:03.681 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-26T01:34:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-26T01:34:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-26T01:34:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-26T01:34:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-26T01:34:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-26T01:34:03.682 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-26T01:34:03.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-26T01:34:03.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-26T01:34:03.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-26T01:34:03.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-26T01:34:03.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-26T01:34:03.683 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-26T01:34:03.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-26T01:34:03.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-26T01:34:03.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-26T01:34:03.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-26T01:34:03.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-26T01:34:03.684 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-26T01:34:03.685 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-26T01:34:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-26T01:34:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-26T01:34:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-26T01:34:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-26T01:34:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-26T01:34:03.686 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-26T01:34:03.687 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-26T01:34:03.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-26T01:34:03.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-26T01:34:03.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-26T01:34:03.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-26T01:34:03.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-26T01:34:03.688 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-26T01:34:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-26T01:34:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-26T01:34:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-26T01:34:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-26T01:34:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-26T01:34:03.689 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-26T01:34:03.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-26T01:34:03.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-26T01:34:03.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-26T01:34:03.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-26T01:34:03.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-26T01:34:03.690 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-26T01:34:03.691 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-26T01:34:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-26T01:34:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-26T01:34:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-26T01:34:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-26T01:34:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-26T01:34:03.692 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-26T01:34:03.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-26T01:34:03.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-26T01:34:03.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-26T01:34:03.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-26T01:34:03.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-26T01:34:03.693 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-26T01:34:03.694 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-26T01:34:03.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-26T01:34:03.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-26T01:34:03.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-26T01:34:03.695 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-26T01:34:03.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-26T01:34:03.697 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-26T01:34:03.698 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-26T01:34:03.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-26T01:34:03.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-26T01:34:03.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-26T01:34:03.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-26T01:34:03.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-26T01:34:03.699 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-26T01:34:03.700 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-26T01:34:03.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-26T01:34:03.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-26T01:34:03.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-26T01:34:03.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-26T01:34:03.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-26T01:34:03.701 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-26T01:34:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-26T01:34:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-26T01:34:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-26T01:34:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-26T01:34:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-26T01:34:03.702 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-26T01:34:03.703 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-26T01:34:03.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-26T01:34:03.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-26T01:34:03.704 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-26T01:34:03.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-26T01:34:03.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-26T01:34:03.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-26T01:34:03.705 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-26T01:34:03.706 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-26T01:34:03.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-26T01:34:03.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-26T01:34:03.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-26T01:34:03.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-26T01:34:03.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-26T01:34:03.707 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-26T01:34:03.708 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-26T01:34:03.709 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-26T01:34:03.710 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-26T01:34:03.711 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-26T01:34:03.712 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-26T01:34:03.713 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-26T01:34:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-26T01:34:03.715 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-26T01:34:03.716 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-26T01:34:03.717 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-26T01:34:03.718 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-26T01:34:03.719 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-26T01:34:03.720 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-26T01:34:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-26T01:34:03.722 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-26T01:34:03.723 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-26T01:34:03.724 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-26T01:34:03.725 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-26T01:34:03.726 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-26T01:34:03.727 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-26T01:34:03.728 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-26T01:34:03.729 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-26T01:34:03.730 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-26T01:34:03.731 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-26T01:34:03.732 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-26T01:34:03.733 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-26T01:34:03.734 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-26T01:34:03.735 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-26T01:34:03.736 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-26T01:34:03.737 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-26T01:34:03.738 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-26T01:34:03.739 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-26T01:34:03.740 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-26T01:34:03.741 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-26T01:34:03.742 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-26T01:34:03.743 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-26T01:34:03.744 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-26T01:34:03.745 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-26T01:34:03.746 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-26T01:34:03.747 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-26T01:34:03.748 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-26T01:34:03.749 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-26T01:34:03.750 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-26T01:34:03.751 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-26T01:34:03.752 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-26T01:34:03.753 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-26T01:34:03.754 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-26T01:34:03.755 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-26T01:34:03.756 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-26T01:34:03.757 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-26T01:34:03.758 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-26T01:34:03.758 INFO:teuthology.orchestra.run.smithi037.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T01:34:03.762 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.762 DEBUG:teuthology.orchestra.run.smithi037:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout:postinstall scriptlet (using /bin/sh): 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout:for file in vmlinuz System.map config; do 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:34:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: fi 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:done 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-26T01:34:03.789 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.790 INFO:teuthology.orchestra.run.smithi037.stdout:fi 2024-04-26T01:34:03.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-04-26T01:34:03.879 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-04-26T01:34:03.879 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-04-26T01:34:03.879 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-04-26T01:34:03.879 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 68.el9 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8264254 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-04-26T01:34:03.880 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:34:03.881 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-04-26T01:34:03.882 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-26T01:34:03.882 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:03.882 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-26T01:34:04.428 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-04-26T01:34:05.839 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-26T01:34:05.859 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-04-26T01:34:05.861 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-04-26T01:34:05.894 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.yKdpDipG5G 2024-04-26T01:34:05.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yKdpDipG5G 2024-04-26T01:34:05.962 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.yKdpDipG5G 2024-04-26T01:34:06.096 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.yKdpDipG5G is 6KB 2024-04-26T01:34:06.147 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.yKdpDipG5G 2024-04-26T01:34:06.162 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-04-26T01:34:06.224 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-26T01:34:06.224 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-26T01:34:06.224 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-26T01:34:06.224 INFO:teuthology.orchestra.run.smithi037.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-26T01:34:06.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-26T01:34:06.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-04-26T01:34:36.377 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-26T01:34:36.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T01:34:36.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:35:36.381 DEBUG:teuthology.orchestra.remote:timed out 2024-04-26T01:35:45.383 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T01:35:45.384 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:35:45.823 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-26T01:35:46.487 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T01:35:46.487 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-26T01:35:46.487 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-04-26T01:35:46.535 INFO:teuthology.orchestra.run.smithi037.stdout:6.9.0-rc3-g806ec36b2112 2024-04-26T01:35:46.536 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-26T01:35:46.536 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-26T01:35:46.536 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-26T01:35:47.540 DEBUG:teuthology.parallel:result is None 2024-04-26T01:35:47.540 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T01:35:47.551 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T01:35:47.551 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:35:47.558 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:35:47.561 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T01:35:47.597 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T01:35:47.605 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T01:35:47.612 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T01:35:47.612 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:35:47.642 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:35:47.645 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T01:35:47.763 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T01:35:47.770 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T01:35:47.770 DEBUG:teuthology.orchestra.run.smithi045:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:35:47.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:35:47.788 DEBUG:teuthology.orchestra.run.smithi071:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T01:35:47.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:35:47.805 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:35:47.811 DEBUG:teuthology.orchestra.run.smithi045:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:35:47.831 DEBUG:teuthology.orchestra.run.smithi071:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T01:35:47.866 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:35:47.887 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:35:47.891 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:35:47.930 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:35:47.942 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:35:48.000 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T01:35:48.003 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T01:35:48.014 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T01:35:48.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:35:48.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:35:48.023 DEBUG:teuthology.orchestra.run.smithi071:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T01:35:48.080 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T01:35:48.089 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T01:35:48.090 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:35:48.094 DEBUG:teuthology.orchestra.run.smithi045:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:35:48.097 DEBUG:teuthology.orchestra.run.smithi071:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T01:35:48.132 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:35:48.246 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:35:48.328 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:35:48.396 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:35:48.474 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:35:48.474 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:35:48.547 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:35:48.593 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:35:48.648 DEBUG:teuthology.orchestra.run.smithi045:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:35:48.734 DEBUG:teuthology.orchestra.run.smithi045:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:35:48.776 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:35:48.776 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:35:48.844 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:35:48.886 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T01:35:48.946 DEBUG:teuthology.orchestra.run.smithi071:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:35:49.029 DEBUG:teuthology.orchestra.run.smithi071:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T01:35:49.073 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:35:49.073 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T01:35:49.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-04-26T01:35:49.146 DEBUG:teuthology.orchestra.run.smithi045:> sudo service rsyslog restart 2024-04-26T01:35:49.148 DEBUG:teuthology.orchestra.run.smithi071:> sudo service rsyslog restart 2024-04-26T01:35:49.183 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:35:49.213 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:35:49.245 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T01:35:49.690 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T01:35:49.698 INFO:teuthology.task.internal:Starting timer... 2024-04-26T01:35:49.698 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T01:35:49.709 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T01:35:49.723 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T01:35:49.723 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-04-26T01:35:49.840 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-04-26T01:35:49.844 DEBUG:teuthology.orchestra.run.smithi045:> sudo service auditd rotate 2024-04-26T01:35:49.914 INFO:teuthology.orchestra.run.smithi045.stdout:Rotating logs: 2024-04-26T01:35:49.916 DEBUG:teuthology.orchestra.run.smithi071:> sudo service auditd rotate 2024-04-26T01:35:49.990 INFO:teuthology.orchestra.run.smithi071.stdout:Rotating logs: 2024-04-26T01:35:49.992 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T01:35:49.992 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-04-26T01:35:50.050 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-04-26T01:35:50.050 DEBUG:teuthology.orchestra.run.smithi045:> /usr/sbin/getenforce 2024-04-26T01:35:50.088 INFO:teuthology.orchestra.run.smithi045.stdout:Permissive 2024-04-26T01:35:50.088 DEBUG:teuthology.orchestra.run.smithi071:> /usr/sbin/getenforce 2024-04-26T01:35:50.132 INFO:teuthology.orchestra.run.smithi071.stdout:Permissive 2024-04-26T01:35:50.132 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi045.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi071.front.sepia.ceph.com': 'permissive'} 2024-04-26T01:35:50.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:35:50.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:35:50.220 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:35:50.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:35:50.247 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T01:35:50.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:35:50.275 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T01:35:50.276 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:35:50.357 DEBUG:teuthology.orchestra.run.smithi045:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:35:50.381 DEBUG:teuthology.orchestra.run.smithi071:> sudo /usr/sbin/setenforce permissive 2024-04-26T01:35:50.408 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T01:35:50.422 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T01:35:50.440 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T01:35:50.442 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi045.front.sepia.ceph.com,smithi071.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T01:41:04.628 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi045.front.sepia.ceph.com'), Remote(name='ubuntu@smithi071.front.sepia.ceph.com')] 2024-04-26T01:41:04.629 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T01:41:04.630 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:41:04.742 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-04-26T01:41:04.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-04-26T01:41:04.900 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-26T01:41:04.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi045.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:41:04.975 DEBUG:teuthology.orchestra.run.smithi045:> true 2024-04-26T01:41:05.055 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi045.front.sepia.ceph.com' 2024-04-26T01:41:05.056 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-26T01:41:05.056 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi071.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T01:41:05.127 DEBUG:teuthology.orchestra.run.smithi071:> true 2024-04-26T01:41:05.204 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi071.front.sepia.ceph.com' 2024-04-26T01:41:05.205 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T01:41:05.215 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T01:41:05.215 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:41:05.215 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:41:05.223 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:41:05.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:41:05.225 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T01:41:05.225 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T01:41:05.258 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:41:05.270 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:41:05.274 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:41:05.283 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:41:05.301 INFO:teuthology.orchestra.run.smithi045.stderr:sudo: ntpd: command not found 2024-04-26T01:41:05.306 INFO:teuthology.orchestra.run.smithi071.stderr:sudo: ntpd: command not found 2024-04-26T01:41:05.312 INFO:teuthology.orchestra.run.smithi045.stdout:506 Cannot talk to daemon 2024-04-26T01:41:05.317 INFO:teuthology.orchestra.run.smithi071.stdout:506 Cannot talk to daemon 2024-04-26T01:41:05.319 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T01:41:05.328 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:41:05.330 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:41:05.342 INFO:teuthology.orchestra.run.smithi045.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:41:05.343 INFO:teuthology.orchestra.run.smithi071.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:41:05.364 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T01:41:05.395 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:41:05.399 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:41:05.399 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-26T01:41:05.399 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.399 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.399 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.400 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.400 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:41:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:41:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-26T01:41:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.403 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.442 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-04-26T01:41:05.476 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-04-26T01:41:05.521 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T01:41:05.564 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T01:41:05.717 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-26T01:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T01:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-26T01:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.725 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T01:41:05.727 INFO:teuthology.run_tasks:Running task install... 2024-04-26T01:41:05.737 DEBUG:teuthology.task.install:project ceph 2024-04-26T01:41:05.737 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-26T01:41:05.737 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-26T01:41:05.737 INFO:teuthology.task.install:Using flavor: default 2024-04-26T01:41:05.744 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T01:41:05.744 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T01:41:05.745 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:41:05.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:41:05.747 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:41:05.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:41:05.749 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-26T01:41:05.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:41:05.900 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:41:05.900 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:41:05.988 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:41:05.989 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:41:05.996 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-26T01:41:05.996 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-26T01:41:06.051 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:41:06.051 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:41:06.051 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:41:06.077 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:41:06.077 DEBUG:teuthology.orchestra.run.smithi071:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:41:06.143 DEBUG:teuthology.orchestra.run.smithi071:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:41:06.156 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:41:06.156 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:41:06.156 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:41:06.218 DEBUG:teuthology.orchestra.run.smithi071:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:41:06.225 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T01:41:06.225 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:41:06.225 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T01:41:06.251 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:41:06.251 DEBUG:teuthology.orchestra.run.smithi045:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:41:06.252 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T01:41:06.252 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T01:41:06.281 INFO:teuthology.orchestra.run.smithi071.stdout:check_obsoletes = 1 2024-04-26T01:41:06.282 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum clean all 2024-04-26T01:41:06.316 DEBUG:teuthology.orchestra.run.smithi045:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:41:06.388 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T01:41:06.390 DEBUG:teuthology.orchestra.run.smithi045:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:41:06.454 INFO:teuthology.orchestra.run.smithi045.stdout:check_obsoletes = 1 2024-04-26T01:41:06.455 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum clean all 2024-04-26T01:41:06.507 INFO:teuthology.orchestra.run.smithi071.stdout:48 files removed 2024-04-26T01:41:06.508 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T01:41:06.533 DEBUG:teuthology.orchestra.run.smithi071:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:41:06.594 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-04-26T01:41:06.600 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-04-26T01:41:06.688 INFO:teuthology.orchestra.run.smithi045.stdout:48 files removed 2024-04-26T01:41:06.715 DEBUG:teuthology.orchestra.run.smithi045:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:41:06.939 INFO:teuthology.orchestra.run.smithi037.stdout:48 files removed 2024-04-26T01:41:06.979 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T01:41:07.313 INFO:teuthology.orchestra.run.smithi071.stdout:ceph packages for x86_64 160 kB/s | 83 kB 00:00 2024-04-26T01:41:07.464 INFO:teuthology.orchestra.run.smithi045.stdout:ceph packages for x86_64 175 kB/s | 83 kB 00:00 2024-04-26T01:41:07.838 INFO:teuthology.orchestra.run.smithi071.stdout:ceph noarch packages 24 kB/s | 12 kB 00:00 2024-04-26T01:41:07.935 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 167 kB/s | 83 kB 00:00 2024-04-26T01:41:08.006 INFO:teuthology.orchestra.run.smithi045.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-04-26T01:41:08.314 INFO:teuthology.orchestra.run.smithi071.stdout:ceph source packages 4.3 kB/s | 1.9 kB 00:00 2024-04-26T01:41:08.390 INFO:teuthology.orchestra.run.smithi045.stdout:ceph source packages 5.5 kB/s | 1.9 kB 00:00 2024-04-26T01:41:08.577 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 21 kB/s | 12 kB 00:00 2024-04-26T01:41:09.008 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T01:41:09.211 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-04-26T01:41:09.643 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - BaseOS 6.6 MB/s | 8.0 MB 00:01 2024-04-26T01:41:10.497 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.0 MB 00:01 2024-04-26T01:41:11.154 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-26T01:41:11.752 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-26T01:41:12.933 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-26T01:41:15.635 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - CRB 6.6 MB/s | 6.2 MB 00:00 2024-04-26T01:41:16.228 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - CRB 7.8 MB/s | 6.2 MB 00:00 2024-04-26T01:41:17.017 INFO:teuthology.orchestra.run.smithi071.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-26T01:41:17.542 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.2 MB 00:00 2024-04-26T01:41:17.893 INFO:teuthology.orchestra.run.smithi045.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-04-26T01:41:19.496 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T01:41:20.615 INFO:teuthology.orchestra.run.smithi071.stdout:Extra Packages for Enterprise Linux 6.0 MB/s | 21 MB 00:03 2024-04-26T01:41:20.974 INFO:teuthology.orchestra.run.smithi045.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 21 MB 00:02 2024-04-26T01:41:25.811 INFO:teuthology.orchestra.run.smithi071.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-04-26T01:41:26.337 INFO:teuthology.orchestra.run.smithi045.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-26T01:41:27.945 INFO:teuthology.orchestra.run.smithi071.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:41:27.945 INFO:teuthology.orchestra.run.smithi071.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:41:27.956 INFO:teuthology.orchestra.run.smithi071.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:41:27.956 INFO:teuthology.orchestra.run.smithi071.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:41:27.957 INFO:teuthology.orchestra.run.smithi071.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:41:27.958 INFO:teuthology.orchestra.run.smithi071.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:41:27.960 INFO:teuthology.orchestra.run.smithi071.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:41:27.962 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:41:27.963 INFO:teuthology.orchestra.run.smithi071.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:41:28.022 INFO:teuthology.orchestra.run.smithi071.stdout:Dependencies resolved. 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: Package Arch Version Repository Size 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout:Installing: 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:41:28.036 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:41:28.037 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:41:28.038 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrading: 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout:Installing dependencies: 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:41:28.039 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:41:28.040 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:41:28.041 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:41:28.042 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:41:28.043 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:41:28.044 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:41:28.045 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:41:28.046 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:41:28.047 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:41:28.048 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:41:28.049 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout:Installing weak dependencies: 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction Summary 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout:======================================================================================= 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout:Install 148 Packages 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout:Upgrade 4 Packages 2024-04-26T01:41:28.050 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:41:28.051 INFO:teuthology.orchestra.run.smithi071.stdout:Total download size: 222 M 2024-04-26T01:41:28.051 INFO:teuthology.orchestra.run.smithi071.stdout:Downloading Packages: 2024-04-26T01:41:28.572 INFO:teuthology.orchestra.run.smithi045.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:41:28.573 INFO:teuthology.orchestra.run.smithi045.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:41:28.584 INFO:teuthology.orchestra.run.smithi045.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:41:28.585 INFO:teuthology.orchestra.run.smithi045.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:41:28.586 INFO:teuthology.orchestra.run.smithi045.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:41:28.586 INFO:teuthology.orchestra.run.smithi045.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:41:28.588 INFO:teuthology.orchestra.run.smithi045.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:41:28.590 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:41:28.591 INFO:teuthology.orchestra.run.smithi045.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:41:28.656 INFO:teuthology.orchestra.run.smithi045.stdout:Dependencies resolved. 2024-04-26T01:41:28.680 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-26T01:41:28.680 INFO:teuthology.orchestra.run.smithi045.stdout: Package Arch Version Repository Size 2024-04-26T01:41:28.680 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-26T01:41:28.680 INFO:teuthology.orchestra.run.smithi045.stdout:Installing: 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:41:28.681 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:41:28.682 INFO:teuthology.orchestra.run.smithi045.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:41:28.683 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrading: 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout:Installing dependencies: 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:41:28.684 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:41:28.685 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:41:28.686 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:41:28.687 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:41:28.688 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:41:28.689 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:41:28.690 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:41:28.691 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:41:28.692 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout:Installing weak dependencies: 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction Summary 2024-04-26T01:41:28.693 INFO:teuthology.orchestra.run.smithi045.stdout:======================================================================================= 2024-04-26T01:41:28.694 INFO:teuthology.orchestra.run.smithi045.stdout:Install 148 Packages 2024-04-26T01:41:28.694 INFO:teuthology.orchestra.run.smithi045.stdout:Upgrade 4 Packages 2024-04-26T01:41:28.694 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:41:28.694 INFO:teuthology.orchestra.run.smithi045.stdout:Total download size: 222 M 2024-04-26T01:41:28.694 INFO:teuthology.orchestra.run.smithi045.stdout:Downloading Packages: 2024-04-26T01:41:28.763 INFO:teuthology.orchestra.run.smithi071.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 30 kB/s | 6.6 kB 00:00 2024-04-26T01:41:29.020 INFO:teuthology.orchestra.run.smithi071.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.4 MB/s | 890 kB 00:00 2024-04-26T01:41:29.071 INFO:teuthology.orchestra.run.smithi071.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.8 MB/s | 145 kB 00:00 2024-04-26T01:41:29.371 INFO:teuthology.orchestra.run.smithi071.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 7.9 MB/s | 2.4 MB 00:00 2024-04-26T01:41:29.471 INFO:teuthology.orchestra.run.smithi071.stdout:(5/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 5.8 MB/s | 5.4 MB 00:00 2024-04-26T01:41:29.588 INFO:teuthology.orchestra.run.smithi071.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 6.9 MB/s | 1.5 MB 00:00 2024-04-26T01:41:29.838 INFO:teuthology.orchestra.run.smithi045.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-26T01:41:29.930 INFO:teuthology.orchestra.run.smithi071.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 4.7 MB 00:00 2024-04-26T01:41:30.714 INFO:teuthology.orchestra.run.smithi045.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.0 MB/s | 890 kB 00:00 2024-04-26T01:41:30.905 INFO:teuthology.orchestra.run.smithi045.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 757 kB/s | 145 kB 00:00 2024-04-26T01:41:30.980 INFO:teuthology.orchestra.run.smithi071.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 10 MB/s | 11 MB 00:01 2024-04-26T01:41:31.030 INFO:teuthology.orchestra.run.smithi071.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 505 kB/s | 25 kB 00:00 2024-04-26T01:41:31.622 INFO:teuthology.orchestra.run.smithi071.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 8.3 MB/s | 17 MB 00:02 2024-04-26T01:41:31.714 INFO:teuthology.orchestra.run.smithi071.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.4 MB/s | 225 kB 00:00 2024-04-26T01:41:31.790 INFO:teuthology.orchestra.run.smithi071.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 361 kB/s | 27 kB 00:00 2024-04-26T01:41:31.865 INFO:teuthology.orchestra.run.smithi071.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 423 kB/s | 32 kB 00:00 2024-04-26T01:41:31.888 INFO:teuthology.orchestra.run.smithi045.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-26T01:41:32.024 INFO:teuthology.orchestra.run.smithi071.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 4.6 MB/s | 747 kB 00:00 2024-04-26T01:41:32.124 INFO:teuthology.orchestra.run.smithi071.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.6 MB/s | 163 kB 00:00 2024-04-26T01:41:32.208 INFO:teuthology.orchestra.run.smithi071.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-26T01:41:32.316 INFO:teuthology.orchestra.run.smithi071.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 4.5 MB/s | 500 kB 00:00 2024-04-26T01:41:32.372 INFO:teuthology.orchestra.run.smithi045.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-26T01:41:32.731 INFO:teuthology.orchestra.run.smithi045.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 1.7 MB/s | 5.4 MB 00:03 2024-04-26T01:41:33.283 INFO:teuthology.orchestra.run.smithi071.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 5.5 MB/s | 5.4 MB 00:00 2024-04-26T01:41:33.484 INFO:teuthology.orchestra.run.smithi071.stdout:(19/152): ceph-common-19.0.0-3156.gd69f13d8.el7 4.3 MB/s | 21 MB 00:04 2024-04-26T01:41:33.509 INFO:teuthology.orchestra.run.smithi071.stdout:(20/152): python3-ceph-argparse-19.0.0-3156.gd6 200 kB/s | 45 kB 00:00 2024-04-26T01:41:33.543 INFO:teuthology.orchestra.run.smithi071.stdout:(21/152): python3-ceph-common-19.0.0-3156.gd69f 2.2 MB/s | 135 kB 00:00 2024-04-26T01:41:33.568 INFO:teuthology.orchestra.run.smithi071.stdout:(22/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.7 MB/s | 161 kB 00:00 2024-04-26T01:41:33.660 INFO:teuthology.orchestra.run.smithi071.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 298 kB 00:00 2024-04-26T01:41:33.727 INFO:teuthology.orchestra.run.smithi071.stdout:(24/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 320 kB 00:00 2024-04-26T01:41:33.752 INFO:teuthology.orchestra.run.smithi071.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.1 MB/s | 99 kB 00:00 2024-04-26T01:41:33.844 INFO:teuthology.orchestra.run.smithi071.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 729 kB/s | 85 kB 00:00 2024-04-26T01:41:33.931 INFO:teuthology.orchestra.run.smithi045.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.0 MB/s | 4.7 MB 00:01 2024-04-26T01:41:33.995 INFO:teuthology.orchestra.run.smithi071.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.1 MB/s | 170 kB 00:00 2024-04-26T01:41:34.136 INFO:teuthology.orchestra.run.smithi071.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 242 kB/s | 34 kB 00:00 2024-04-26T01:41:34.254 INFO:teuthology.orchestra.run.smithi071.stdout:(29/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 6.2 MB/s | 3.1 MB 00:00 2024-04-26T01:41:34.279 INFO:teuthology.orchestra.run.smithi071.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.0 MB/s | 142 kB 00:00 2024-04-26T01:41:35.029 INFO:teuthology.orchestra.run.smithi071.stdout:(31/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 12 MB/s | 49 MB 00:03 2024-04-26T01:41:35.088 INFO:teuthology.orchestra.run.smithi071.stdout:(32/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.2 MB/s | 3.5 MB 00:00 2024-04-26T01:41:35.113 INFO:teuthology.orchestra.run.smithi071.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.9 MB/s | 249 kB 00:00 2024-04-26T01:41:35.131 INFO:teuthology.orchestra.run.smithi071.stdout:(34/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 49 kB 00:00 2024-04-26T01:41:35.147 INFO:teuthology.orchestra.run.smithi071.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3156.gd 493 kB/s | 16 kB 00:00 2024-04-26T01:41:35.205 INFO:teuthology.orchestra.run.smithi071.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.5 MB/s | 273 kB 00:00 2024-04-26T01:41:35.239 INFO:teuthology.orchestra.run.smithi071.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 8.1 MB/s | 762 kB 00:00 2024-04-26T01:41:35.264 INFO:teuthology.orchestra.run.smithi071.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 311 kB/s | 18 kB 00:00 2024-04-26T01:41:35.431 INFO:teuthology.orchestra.run.smithi071.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 419 kB/s | 80 kB 00:00 2024-04-26T01:41:35.498 INFO:teuthology.orchestra.run.smithi071.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-26T01:41:35.549 INFO:teuthology.orchestra.run.smithi071.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-04-26T01:41:35.607 INFO:teuthology.orchestra.run.smithi071.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.3 MB/s | 253 kB 00:00 2024-04-26T01:41:35.708 INFO:teuthology.orchestra.run.smithi071.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 7.8 MB/s | 1.2 MB 00:00 2024-04-26T01:41:35.733 INFO:teuthology.orchestra.run.smithi071.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 853 kB/s | 106 kB 00:00 2024-04-26T01:41:35.775 INFO:teuthology.orchestra.run.smithi071.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-26T01:41:35.800 INFO:teuthology.orchestra.run.smithi071.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-26T01:41:35.842 INFO:teuthology.orchestra.run.smithi071.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-04-26T01:41:35.918 INFO:teuthology.orchestra.run.smithi071.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.7 MB/s | 689 kB 00:00 2024-04-26T01:41:36.010 INFO:teuthology.orchestra.run.smithi071.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-04-26T01:41:36.035 INFO:teuthology.orchestra.run.smithi071.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 918 kB/s | 107 kB 00:00 2024-04-26T01:41:36.127 INFO:teuthology.orchestra.run.smithi071.stdout:(51/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-26T01:41:36.152 INFO:teuthology.orchestra.run.smithi071.stdout:(52/152): elfutils-libelf-devel-0.190-2.el9.x86 151 kB/s | 21 kB 00:00 2024-04-26T01:41:36.177 INFO:teuthology.orchestra.run.smithi071.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-26T01:41:36.202 INFO:teuthology.orchestra.run.smithi071.stdout:(54/152): flexiblas-3.0.4-8.el9.x86_64.rpm 438 kB/s | 33 kB 00:00 2024-04-26T01:41:36.245 INFO:teuthology.orchestra.run.smithi071.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-26T01:41:36.264 INFO:teuthology.orchestra.run.smithi045.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 3.1 MB/s | 21 MB 00:06 2024-04-26T01:41:36.298 INFO:teuthology.orchestra.run.smithi045.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 763 kB/s | 25 kB 00:00 2024-04-26T01:41:36.412 INFO:teuthology.orchestra.run.smithi071.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 53 kB/s | 11 kB 00:00 2024-04-26T01:41:36.515 INFO:teuthology.orchestra.run.smithi045.stdout:(10/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 4.4 MB/s | 17 MB 00:03 2024-04-26T01:41:36.545 INFO:teuthology.orchestra.run.smithi071.stdout:(57/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.1 MB/s | 3.0 MB 00:00 2024-04-26T01:41:36.571 INFO:teuthology.orchestra.run.smithi071.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 493 kB/s | 160 kB 00:00 2024-04-26T01:41:36.581 INFO:teuthology.orchestra.run.smithi045.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 3.3 MB/s | 225 kB 00:00 2024-04-26T01:41:36.595 INFO:teuthology.orchestra.run.smithi071.stdout:(59/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 247 kB/s | 45 kB 00:00 2024-04-26T01:41:36.615 INFO:teuthology.orchestra.run.smithi045.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 814 kB/s | 27 kB 00:00 2024-04-26T01:41:36.648 INFO:teuthology.orchestra.run.smithi045.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 957 kB/s | 32 kB 00:00 2024-04-26T01:41:36.671 INFO:teuthology.orchestra.run.smithi071.stdout:(60/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-26T01:41:36.688 INFO:teuthology.orchestra.run.smithi071.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 278 kB/s | 25 kB 00:00 2024-04-26T01:41:36.730 INFO:teuthology.orchestra.run.smithi071.stdout:(62/152): libtool-2.4.6-45.el9.x86_64.rpm 4.3 MB/s | 582 kB 00:00 2024-04-26T01:41:36.755 INFO:teuthology.orchestra.run.smithi071.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-26T01:41:36.765 INFO:teuthology.orchestra.run.smithi045.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 6.3 MB/s | 747 kB 00:00 2024-04-26T01:41:36.780 INFO:teuthology.orchestra.run.smithi071.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-04-26T01:41:36.816 INFO:teuthology.orchestra.run.smithi045.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 3.2 MB/s | 163 kB 00:00 2024-04-26T01:41:36.822 INFO:teuthology.orchestra.run.smithi071.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T01:41:36.856 INFO:teuthology.orchestra.run.smithi071.stdout:(66/152): openblas-0.3.21-2.el9.x86_64.rpm 351 kB/s | 35 kB 00:00 2024-04-26T01:41:36.866 INFO:teuthology.orchestra.run.smithi045.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 2.5 MB/s | 126 kB 00:00 2024-04-26T01:41:36.958 INFO:teuthology.orchestra.run.smithi045.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 5.3 MB/s | 500 kB 00:00 2024-04-26T01:41:37.291 INFO:teuthology.orchestra.run.smithi045.stdout:(18/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 3.2 MB/s | 11 MB 00:03 2024-04-26T01:41:37.392 INFO:teuthology.orchestra.run.smithi045.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 452 kB/s | 45 kB 00:00 2024-04-26T01:41:37.448 INFO:teuthology.orchestra.run.smithi071.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 2.7 MB/s | 1.6 MB 00:00 2024-04-26T01:41:37.500 INFO:teuthology.orchestra.run.smithi045.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.2 MB/s | 135 kB 00:00 2024-04-26T01:41:37.582 INFO:teuthology.orchestra.run.smithi071.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 5.9 MB/s | 4.7 MB 00:00 2024-04-26T01:41:37.609 INFO:teuthology.orchestra.run.smithi045.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-26T01:41:37.726 INFO:teuthology.orchestra.run.smithi045.stdout:(22/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 7.0 MB/s | 5.4 MB 00:00 2024-04-26T01:41:37.757 INFO:teuthology.orchestra.run.smithi071.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 4.7 MB/s | 4.4 MB 00:00 2024-04-26T01:41:37.793 INFO:teuthology.orchestra.run.smithi045.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 4.4 MB/s | 298 kB 00:00 2024-04-26T01:41:37.807 INFO:teuthology.orchestra.run.smithi071.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 2.5 MB/s | 911 kB 00:00 2024-04-26T01:41:37.832 INFO:teuthology.orchestra.run.smithi071.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 534 kB/s | 133 kB 00:00 2024-04-26T01:41:37.843 INFO:teuthology.orchestra.run.smithi045.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 99 kB 00:00 2024-04-26T01:41:37.877 INFO:teuthology.orchestra.run.smithi045.stdout:(25/152): python3-rados-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 320 kB 00:00 2024-04-26T01:41:37.902 INFO:teuthology.orchestra.run.smithi045.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.4 MB/s | 85 kB 00:00 2024-04-26T01:41:37.952 INFO:teuthology.orchestra.run.smithi045.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 3.3 MB/s | 170 kB 00:00 2024-04-26T01:41:38.003 INFO:teuthology.orchestra.run.smithi045.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 680 kB/s | 34 kB 00:00 2024-04-26T01:41:38.153 INFO:teuthology.orchestra.run.smithi045.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 950 kB/s | 142 kB 00:00 2024-04-26T01:41:38.158 INFO:teuthology.orchestra.run.smithi071.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 270 kB/s | 87 kB 00:00 2024-04-26T01:41:38.191 INFO:teuthology.orchestra.run.smithi071.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-04-26T01:41:38.358 INFO:teuthology.orchestra.run.smithi071.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 9.6 MB/s | 5.8 MB 00:00 2024-04-26T01:41:38.467 INFO:teuthology.orchestra.run.smithi071.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 891 kB/s | 244 kB 00:00 2024-04-26T01:41:38.537 INFO:teuthology.orchestra.run.smithi045.stdout:(30/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 9.2 MB/s | 3.5 MB 00:00 2024-04-26T01:41:38.594 INFO:teuthology.orchestra.run.smithi071.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.0 MB/s | 248 kB 00:00 2024-04-26T01:41:38.678 INFO:teuthology.orchestra.run.smithi071.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 11 MB/s | 6.0 MB 00:00 2024-04-26T01:41:38.703 INFO:teuthology.orchestra.run.smithi071.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 109 kB/s | 26 kB 00:00 2024-04-26T01:41:38.728 INFO:teuthology.orchestra.run.smithi071.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 144 kB/s | 19 kB 00:00 2024-04-26T01:41:38.753 INFO:teuthology.orchestra.run.smithi071.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-04-26T01:41:38.779 INFO:teuthology.orchestra.run.smithi071.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T01:41:38.862 INFO:teuthology.orchestra.run.smithi071.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-26T01:41:38.938 INFO:teuthology.orchestra.run.smithi071.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-26T01:41:39.105 INFO:teuthology.orchestra.run.smithi071.stdout:(84/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 14 MB/s | 5.1 MB 00:00 2024-04-26T01:41:39.130 INFO:teuthology.orchestra.run.smithi071.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 290 kB/s | 77 kB 00:00 2024-04-26T01:41:39.155 INFO:teuthology.orchestra.run.smithi071.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 195 kB/s | 42 kB 00:00 2024-04-26T01:41:39.189 INFO:teuthology.orchestra.run.smithi071.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-26T01:41:39.231 INFO:teuthology.orchestra.run.smithi071.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-04-26T01:41:39.256 INFO:teuthology.orchestra.run.smithi071.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-26T01:41:39.281 INFO:teuthology.orchestra.run.smithi071.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 551 kB/s | 51 kB 00:00 2024-04-26T01:41:39.286 INFO:teuthology.orchestra.run.smithi045.stdout:(31/152): ceph-mgr-diskprediction-local-19.0.0- 9.8 MB/s | 7.4 MB 00:00 2024-04-26T01:41:39.306 INFO:teuthology.orchestra.run.smithi071.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 715 kB/s | 54 kB 00:00 2024-04-26T01:41:39.337 INFO:teuthology.orchestra.run.smithi045.stdout:(32/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.1 MB/s | 3.1 MB 00:01 2024-04-26T01:41:39.362 INFO:teuthology.orchestra.run.smithi045.stdout:(33/152): ceph-mgr-modules-core-19.0.0-3156.gd6 3.2 MB/s | 249 kB 00:00 2024-04-26T01:41:39.404 INFO:teuthology.orchestra.run.smithi045.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 396 kB/s | 16 kB 00:00 2024-04-26T01:41:39.438 INFO:teuthology.orchestra.run.smithi045.stdout:(35/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 491 kB/s | 49 kB 00:00 2024-04-26T01:41:39.448 INFO:teuthology.orchestra.run.smithi071.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 250 kB/s | 42 kB 00:00 2024-04-26T01:41:39.465 INFO:teuthology.orchestra.run.smithi071.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 279 kB/s | 44 kB 00:00 2024-04-26T01:41:39.488 INFO:teuthology.orchestra.run.smithi045.stdout:(36/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-26T01:41:39.530 INFO:teuthology.orchestra.run.smithi045.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 439 kB/s | 18 kB 00:00 2024-04-26T01:41:39.613 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-26T01:41:39.657 INFO:teuthology.orchestra.run.smithi071.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-04-26T01:41:39.672 INFO:teuthology.orchestra.run.smithi045.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 567 kB/s | 80 kB 00:00 2024-04-26T01:41:39.674 INFO:teuthology.orchestra.run.smithi071.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 173 kB/s | 36 kB 00:00 2024-04-26T01:41:39.747 INFO:teuthology.orchestra.run.smithi045.stdout:(39/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.4 MB/s | 762 kB 00:00 2024-04-26T01:41:39.822 INFO:teuthology.orchestra.run.smithi045.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 5.2 MB/s | 803 kB 00:00 2024-04-26T01:41:39.848 INFO:teuthology.orchestra.run.smithi045.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-26T01:41:39.849 INFO:teuthology.orchestra.run.smithi071.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 248 kB/s | 47 kB 00:00 2024-04-26T01:41:39.898 INFO:teuthology.orchestra.run.smithi045.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-04-26T01:41:39.908 INFO:teuthology.orchestra.run.smithi071.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-04-26T01:41:39.965 INFO:teuthology.orchestra.run.smithi045.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-04-26T01:41:39.990 INFO:teuthology.orchestra.run.smithi045.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-26T01:41:40.007 INFO:teuthology.orchestra.run.smithi045.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-26T01:41:40.040 INFO:teuthology.orchestra.run.smithi045.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T01:41:40.066 INFO:teuthology.orchestra.run.smithi045.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-26T01:41:40.075 INFO:teuthology.orchestra.run.smithi071.stdout:(98/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-04-26T01:41:40.100 INFO:teuthology.orchestra.run.smithi071.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 232 kB/s | 58 kB 00:00 2024-04-26T01:41:40.108 INFO:teuthology.orchestra.run.smithi045.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 10 MB/s | 689 kB 00:00 2024-04-26T01:41:40.126 INFO:teuthology.orchestra.run.smithi071.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 434 kB/s | 94 kB 00:00 2024-04-26T01:41:40.141 INFO:teuthology.orchestra.run.smithi045.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-26T01:41:40.159 INFO:teuthology.orchestra.run.smithi071.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 769 kB/s | 45 kB 00:00 2024-04-26T01:41:40.166 INFO:teuthology.orchestra.run.smithi045.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.8 MB/s | 107 kB 00:00 2024-04-26T01:41:40.183 INFO:teuthology.orchestra.run.smithi045.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 520 kB/s | 21 kB 00:00 2024-04-26T01:41:40.185 INFO:teuthology.orchestra.run.smithi071.stdout:(102/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 761 kB/s | 64 kB 00:00 2024-04-26T01:41:40.210 INFO:teuthology.orchestra.run.smithi071.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-04-26T01:41:40.225 INFO:teuthology.orchestra.run.smithi045.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-26T01:41:40.243 INFO:teuthology.orchestra.run.smithi071.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-26T01:41:40.250 INFO:teuthology.orchestra.run.smithi045.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 489 kB/s | 33 kB 00:00 2024-04-26T01:41:40.317 INFO:teuthology.orchestra.run.smithi045.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-04-26T01:41:40.376 INFO:teuthology.orchestra.run.smithi045.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-26T01:41:40.402 INFO:teuthology.orchestra.run.smithi045.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 136 kB/s | 11 kB 00:00 2024-04-26T01:41:40.435 INFO:teuthology.orchestra.run.smithi045.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T01:41:40.460 INFO:teuthology.orchestra.run.smithi045.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 774 kB/s | 45 kB 00:00 2024-04-26T01:41:40.519 INFO:teuthology.orchestra.run.smithi045.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2024-04-26T01:41:40.544 INFO:teuthology.orchestra.run.smithi045.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-26T01:41:40.586 INFO:teuthology.orchestra.run.smithi045.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 386 kB/s | 25 kB 00:00 2024-04-26T01:41:40.612 INFO:teuthology.orchestra.run.smithi045.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 709 kB/s | 47 kB 00:00 2024-04-26T01:41:40.670 INFO:teuthology.orchestra.run.smithi045.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-26T01:41:40.769 INFO:teuthology.orchestra.run.smithi071.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 37 kB/s | 19 kB 00:00 2024-04-26T01:41:40.787 INFO:teuthology.orchestra.run.smithi045.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-26T01:41:40.812 INFO:teuthology.orchestra.run.smithi045.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 249 kB/s | 35 kB 00:00 2024-04-26T01:41:40.896 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ziply.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 50.47.0.53) 2024-04-26T01:41:41.126 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ziply.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 50.47.0.53) 2024-04-26T01:41:41.221 INFO:teuthology.orchestra.run.smithi045.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 12 MB/s | 4.7 MB 00:00 2024-04-26T01:41:41.227 INFO:teuthology.orchestra.run.smithi071.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 304 kB/s | 309 kB 00:01 2024-04-26T01:41:41.228 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:41.288 INFO:teuthology.orchestra.run.smithi045.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.3 MB/s | 4.4 MB 00:00 2024-04-26T01:41:41.320 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://ziply.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 50.47.0.53) 2024-04-26T01:41:41.378 INFO:teuthology.orchestra.run.smithi071.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 470 kB/s | 548 kB 00:01 2024-04-26T01:41:41.378 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:41.407 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://ziply.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 50.47.0.53) 2024-04-26T01:41:41.442 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 208.68.237.5) 2024-04-26T01:41:41.495 INFO:teuthology.orchestra.run.smithi071.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 416 kB/s | 49 kB 00:00 2024-04-26T01:41:41.495 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 208.68.237.5) 2024-04-26T01:41:41.629 INFO:teuthology.orchestra.run.smithi071.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 505 kB/s | 67 kB 00:00 2024-04-26T01:41:41.679 INFO:teuthology.orchestra.run.smithi071.stdout:(110/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 58 kB/s | 26 kB 00:00 2024-04-26T01:41:41.779 INFO:teuthology.orchestra.run.smithi071.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.0 MB/s | 151 kB 00:00 2024-04-26T01:41:41.780 INFO:teuthology.orchestra.run.smithi045.stdout:(68/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 9.0 MB/s | 49 MB 00:05 2024-04-26T01:41:41.780 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ziply.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 50.47.0.53) 2024-04-26T01:41:41.805 INFO:teuthology.orchestra.run.smithi045.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-04-26T01:41:41.838 INFO:teuthology.orchestra.run.smithi045.stdout:(70/152): postgresql-13.14-1.el9.x86_64.rpm 2.9 MB/s | 1.6 MB 00:00 2024-04-26T01:41:41.880 INFO:teuthology.orchestra.run.smithi045.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-04-26T01:41:41.903 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ziply.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 50.47.0.53) 2024-04-26T01:41:41.976 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:42.009 INFO:teuthology.orchestra.run.smithi071.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:42.014 INFO:teuthology.orchestra.run.smithi045.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 674 kB/s | 87 kB 00:00 2024-04-26T01:41:42.079 INFO:teuthology.orchestra.run.smithi071.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 1.8 MB/s | 548 kB 00:00 2024-04-26T01:41:42.114 INFO:teuthology.orchestra.run.smithi045.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-26T01:41:42.148 INFO:teuthology.orchestra.run.smithi045.stdout:(74/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.8 MB/s | 1.0 MB 00:00 2024-04-26T01:41:42.155 INFO:teuthology.orchestra.run.smithi071.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-26T01:41:42.197 INFO:teuthology.orchestra.run.smithi071.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 371 kB/s | 43 kB 00:00 2024-04-26T01:41:42.264 INFO:teuthology.orchestra.run.smithi045.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-04-26T01:41:42.272 INFO:teuthology.orchestra.run.smithi071.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-04-26T01:41:42.289 INFO:teuthology.orchestra.run.smithi045.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.7 MB/s | 248 kB 00:00 2024-04-26T01:41:42.306 INFO:teuthology.orchestra.run.smithi071.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-26T01:41:42.315 INFO:teuthology.orchestra.run.smithi045.stdout:(77/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-26T01:41:42.332 INFO:teuthology.orchestra.run.smithi045.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 385 kB/s | 26 kB 00:00 2024-04-26T01:41:42.348 INFO:teuthology.orchestra.run.smithi045.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 329 kB/s | 19 kB 00:00 2024-04-26T01:41:42.364 INFO:teuthology.orchestra.run.smithi071.stdout:(117/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.2 MB/s | 838 kB 00:00 2024-04-26T01:41:42.432 INFO:teuthology.orchestra.run.smithi045.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 347 kB/s | 35 kB 00:00 2024-04-26T01:41:42.457 INFO:teuthology.orchestra.run.smithi045.stdout:(81/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-26T01:41:42.464 INFO:teuthology.orchestra.run.smithi071.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 893 kB/s | 172 kB 00:00 2024-04-26T01:41:42.489 INFO:teuthology.orchestra.run.smithi071.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 1.9 MB/s | 358 kB 00:00 2024-04-26T01:41:42.541 INFO:teuthology.orchestra.run.smithi045.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-04-26T01:41:42.567 INFO:teuthology.orchestra.run.smithi045.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.9 MB/s | 250 kB 00:00 2024-04-26T01:41:42.582 INFO:teuthology.orchestra.run.smithi071.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 302 kB/s | 35 kB 00:00 2024-04-26T01:41:42.592 INFO:teuthology.orchestra.run.smithi045.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.7 MB/s | 222 kB 00:00 2024-04-26T01:41:42.608 INFO:teuthology.orchestra.run.smithi045.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-26T01:41:42.625 INFO:teuthology.orchestra.run.smithi071.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.5 MB/s | 206 kB 00:00 2024-04-26T01:41:42.626 INFO:teuthology.orchestra.run.smithi045.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-04-26T01:41:42.651 INFO:teuthology.orchestra.run.smithi045.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-26T01:41:42.676 INFO:teuthology.orchestra.run.smithi045.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-26T01:41:42.710 INFO:teuthology.orchestra.run.smithi045.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-26T01:41:42.726 INFO:teuthology.orchestra.run.smithi045.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 682 kB/s | 51 kB 00:00 2024-04-26T01:41:42.743 INFO:teuthology.orchestra.run.smithi045.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-04-26T01:41:42.757 INFO:teuthology.orchestra.run.smithi071.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-26T01:41:42.768 INFO:teuthology.orchestra.run.smithi045.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-26T01:41:42.802 INFO:teuthology.orchestra.run.smithi045.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 774 kB/s | 44 kB 00:00 2024-04-26T01:41:42.807 INFO:teuthology.orchestra.run.smithi071.stdout:(123/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 684 kB/s | 303 kB 00:00 2024-04-26T01:41:42.853 INFO:teuthology.orchestra.run.smithi045.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-26T01:41:42.883 INFO:teuthology.orchestra.run.smithi071.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-26T01:41:42.886 INFO:teuthology.orchestra.run.smithi045.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 441 kB/s | 36 kB 00:00 2024-04-26T01:41:42.933 INFO:teuthology.orchestra.run.smithi071.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 141 kB/s | 18 kB 00:00 2024-04-26T01:41:42.953 INFO:teuthology.orchestra.run.smithi045.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 488 kB/s | 47 kB 00:00 2024-04-26T01:41:43.017 INFO:teuthology.orchestra.run.smithi071.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 173 kB/s | 23 kB 00:00 2024-04-26T01:41:43.020 INFO:teuthology.orchestra.run.smithi045.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.3 MB/s | 305 kB 00:00 2024-04-26T01:41:43.075 INFO:teuthology.orchestra.run.smithi071.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 4.1 MB/s | 2.0 MB 00:00 2024-04-26T01:41:43.079 INFO:teuthology.orchestra.run.smithi045.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 467 kB/s | 58 kB 00:00 2024-04-26T01:41:43.101 INFO:teuthology.orchestra.run.smithi071.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 116 kB/s | 19 kB 00:00 2024-04-26T01:41:43.126 INFO:teuthology.orchestra.run.smithi071.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-26T01:41:43.226 INFO:teuthology.orchestra.run.smithi071.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-04-26T01:41:43.243 INFO:teuthology.orchestra.run.smithi071.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 326 kB/s | 46 kB 00:00 2024-04-26T01:41:43.246 INFO:teuthology.orchestra.run.smithi045.stdout:(99/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 30 MB/s | 16 MB 00:00 2024-04-26T01:41:43.269 INFO:teuthology.orchestra.run.smithi071.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 552 kB/s | 79 kB 00:00 2024-04-26T01:41:43.271 INFO:teuthology.orchestra.run.smithi045.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 378 kB/s | 94 kB 00:00 2024-04-26T01:41:43.296 INFO:teuthology.orchestra.run.smithi045.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 294 kB/s | 64 kB 00:00 2024-04-26T01:41:43.321 INFO:teuthology.orchestra.run.smithi045.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 448 kB/s | 22 kB 00:00 2024-04-26T01:41:43.327 INFO:teuthology.orchestra.run.smithi071.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 573 kB/s | 58 kB 00:00 2024-04-26T01:41:43.346 INFO:teuthology.orchestra.run.smithi045.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 597 kB/s | 45 kB 00:00 2024-04-26T01:41:43.369 INFO:teuthology.orchestra.run.smithi071.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-26T01:41:43.388 INFO:teuthology.orchestra.run.smithi045.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.2 MB/s | 863 kB 00:00 2024-04-26T01:41:43.420 INFO:teuthology.orchestra.run.smithi071.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-26T01:41:43.445 INFO:teuthology.orchestra.run.smithi071.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 768 kB/s | 90 kB 00:00 2024-04-26T01:41:43.479 INFO:teuthology.orchestra.run.smithi071.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-04-26T01:41:43.480 INFO:teuthology.orchestra.run.smithi045.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-26T01:41:43.504 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://uvermont.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 132.198.255.58) 2024-04-26T01:41:43.555 INFO:teuthology.orchestra.run.smithi045.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.5 MB/s | 309 kB 00:00 2024-04-26T01:41:43.573 INFO:teuthology.orchestra.run.smithi071.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 201 kB/s | 31 kB 00:00 2024-04-26T01:41:43.604 INFO:teuthology.orchestra.run.smithi071.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-26T01:41:43.614 INFO:teuthology.orchestra.run.smithi045.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-04-26T01:41:43.615 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://uvermont.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 132.198.255.58) 2024-04-26T01:41:43.629 INFO:teuthology.orchestra.run.smithi071.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 392 kB/s | 59 kB 00:00 2024-04-26T01:41:43.638 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://uvermont.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 132.198.255.58) 2024-04-26T01:41:43.688 INFO:teuthology.orchestra.run.smithi071.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 312 kB/s | 36 kB 00:00 2024-04-26T01:41:43.689 INFO:teuthology.orchestra.run.smithi045.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 656 kB/s | 49 kB 00:00 2024-04-26T01:41:43.714 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://uvermont.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 132.198.255.58) 2024-04-26T01:41:43.722 INFO:teuthology.orchestra.run.smithi071.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 434 kB/s | 51 kB 00:00 2024-04-26T01:41:43.748 INFO:teuthology.orchestra.run.smithi045.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-26T01:41:43.772 INFO:teuthology.orchestra.run.smithi071.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.6 MB/s | 230 kB 00:00 2024-04-26T01:41:43.814 INFO:teuthology.orchestra.run.smithi071.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 713 kB/s | 90 kB 00:00 2024-04-26T01:41:43.823 INFO:teuthology.orchestra.run.smithi045.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 3.1 MB/s | 151 kB 00:00 2024-04-26T01:41:43.856 INFO:teuthology.orchestra.run.smithi071.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-04-26T01:41:43.862 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://uvermont.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 132.198.255.58) 2024-04-26T01:41:43.886 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://uvermont.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 132.198.255.58) 2024-04-26T01:41:43.889 INFO:teuthology.orchestra.run.smithi071.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-04-26T01:41:43.911 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://southfront.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 147.160.170.17) 2024-04-26T01:41:43.956 INFO:teuthology.orchestra.run.smithi071.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.3 MB/s | 192 kB 00:00 2024-04-26T01:41:43.962 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://southfront.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 147.160.170.17) 2024-04-26T01:41:44.009 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://southfront.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 147.160.170.17) 2024-04-26T01:41:44.032 INFO:teuthology.orchestra.run.smithi071.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-26T01:41:44.056 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://southfront.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 147.160.170.17) 2024-04-26T01:41:44.056 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://southfront.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 147.160.170.17) 2024-04-26T01:41:44.130 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://southfront.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 147.160.170.17) 2024-04-26T01:41:44.132 INFO:teuthology.orchestra.run.smithi071.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 14 MB/s | 1.3 MB 00:00 2024-04-26T01:41:44.212 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://nocix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 69.30.208.233) 2024-04-26T01:41:44.231 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://nocix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 69.30.208.233) 2024-04-26T01:41:44.274 INFO:teuthology.orchestra.run.smithi071.stdout:(150/152): librados2-19.0.0-3156.gd69f13d8.el7. 8.8 MB/s | 3.4 MB 00:00 2024-04-26T01:41:44.310 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://nocix.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 69.30.208.233) 2024-04-26T01:41:44.310 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://nocix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 69.30.208.233) 2024-04-26T01:41:44.324 INFO:teuthology.orchestra.run.smithi071.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-26T01:41:44.329 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://nocix.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 69.30.208.233) 2024-04-26T01:41:44.359 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://nocix.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 69.30.208.233) 2024-04-26T01:41:44.378 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:44.392 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 208.68.237.5) 2024-04-26T01:41:44.471 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:44.529 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:44.534 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 208.68.237.5) 2024-04-26T01:41:44.616 INFO:teuthology.orchestra.run.smithi071.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.8 MB/s | 3.1 MB 00:00 2024-04-26T01:41:44.623 INFO:teuthology.orchestra.run.smithi071.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:41:44.623 INFO:teuthology.orchestra.run.smithi071.stdout:Total 13 MB/s | 222 MB 00:16 2024-04-26T01:41:44.635 INFO:teuthology.orchestra.run.smithi045.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://volico.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 208.68.237.5) 2024-04-26T01:41:44.715 INFO:teuthology.orchestra.run.smithi045.stdout:(111/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 23 kB/s | 26 kB 00:01 2024-04-26T01:41:44.923 INFO:teuthology.orchestra.run.smithi045.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-04-26T01:41:44.974 INFO:teuthology.orchestra.run.smithi045.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 874 kB/s | 43 kB 00:00 2024-04-26T01:41:45.024 INFO:teuthology.orchestra.run.smithi045.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-26T01:41:45.074 INFO:teuthology.orchestra.run.smithi045.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-04-26T01:41:45.099 INFO:teuthology.orchestra.run.smithi045.stdout:(116/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 657 kB/s | 838 kB 00:01 2024-04-26T01:41:45.125 INFO:teuthology.orchestra.run.smithi045.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-26T01:41:45.192 INFO:teuthology.orchestra.run.smithi045.stdout:(118/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.6 MB/s | 4.4 MB 00:01 2024-04-26T01:41:45.217 INFO:teuthology.orchestra.run.smithi045.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-26T01:41:45.267 INFO:teuthology.orchestra.run.smithi045.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-26T01:41:45.301 INFO:teuthology.orchestra.run.smithi045.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-04-26T01:41:45.335 INFO:teuthology.orchestra.run.smithi045.stdout:(122/152): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-04-26T01:41:45.385 INFO:teuthology.orchestra.run.smithi045.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 215 kB/s | 11 kB 00:00 2024-04-26T01:41:45.418 INFO:teuthology.orchestra.run.smithi045.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 1.2 MB/s | 144 kB 00:00 2024-04-26T01:41:45.461 INFO:teuthology.orchestra.run.smithi045.stdout:(125/152): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-04-26T01:41:45.486 INFO:teuthology.orchestra.run.smithi045.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2024-04-26T01:41:45.519 INFO:teuthology.orchestra.run.smithi045.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-04-26T01:41:45.545 INFO:teuthology.orchestra.run.smithi045.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 232 kB/s | 19 kB 00:00 2024-04-26T01:41:45.570 INFO:teuthology.orchestra.run.smithi045.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 235 kB/s | 20 kB 00:00 2024-04-26T01:41:45.595 INFO:teuthology.orchestra.run.smithi045.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 924 kB/s | 46 kB 00:00 2024-04-26T01:41:45.629 INFO:teuthology.orchestra.run.smithi045.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-04-26T01:41:45.654 INFO:teuthology.orchestra.run.smithi045.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 939 kB/s | 79 kB 00:00 2024-04-26T01:41:45.679 INFO:teuthology.orchestra.run.smithi045.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 690 kB/s | 58 kB 00:00 2024-04-26T01:41:45.696 INFO:teuthology.orchestra.run.smithi045.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 393 kB/s | 16 kB 00:00 2024-04-26T01:41:45.713 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction check 2024-04-26T01:41:45.721 INFO:teuthology.orchestra.run.smithi045.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-04-26T01:41:45.746 INFO:teuthology.orchestra.run.smithi045.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-26T01:41:45.772 INFO:teuthology.orchestra.run.smithi045.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 375 kB/s | 28 kB 00:00 2024-04-26T01:41:45.797 INFO:teuthology.orchestra.run.smithi045.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 409 kB/s | 31 kB 00:00 2024-04-26T01:41:45.804 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-26T01:41:45.822 INFO:teuthology.orchestra.run.smithi045.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-26T01:41:45.847 INFO:teuthology.orchestra.run.smithi045.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-04-26T01:41:45.856 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction check succeeded. 2024-04-26T01:41:45.856 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction test 2024-04-26T01:41:45.872 INFO:teuthology.orchestra.run.smithi045.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 475 kB/s | 36 kB 00:00 2024-04-26T01:41:45.898 INFO:teuthology.orchestra.run.smithi045.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 678 kB/s | 51 kB 00:00 2024-04-26T01:41:45.915 INFO:teuthology.orchestra.run.smithi045.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-04-26T01:41:45.940 INFO:teuthology.orchestra.run.smithi045.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-04-26T01:41:45.965 INFO:teuthology.orchestra.run.smithi045.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-26T01:41:45.990 INFO:teuthology.orchestra.run.smithi045.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-26T01:41:46.016 INFO:teuthology.orchestra.run.smithi045.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-26T01:41:46.057 INFO:teuthology.orchestra.run.smithi045.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T01:41:46.266 INFO:teuthology.orchestra.run.smithi045.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 8.9 MB/s | 1.3 MB 00:00 2024-04-26T01:41:46.341 INFO:teuthology.orchestra.run.smithi045.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-04-26T01:41:46.717 INFO:teuthology.orchestra.run.smithi045.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 4.7 MB/s | 3.4 MB 00:00 2024-04-26T01:41:46.808 INFO:teuthology.orchestra.run.smithi045.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.0 MB/s | 3.1 MB 00:00 2024-04-26T01:41:46.816 INFO:teuthology.orchestra.run.smithi045.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:41:46.816 INFO:teuthology.orchestra.run.smithi045.stdout:Total 12 MB/s | 222 MB 00:18 2024-04-26T01:41:47.624 INFO:teuthology.orchestra.run.smithi071.stdout:Transaction test succeeded. 2024-04-26T01:41:47.625 INFO:teuthology.orchestra.run.smithi071.stdout:Running transaction 2024-04-26T01:41:47.949 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction check 2024-04-26T01:41:48.112 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction check succeeded. 2024-04-26T01:41:48.112 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction test 2024-04-26T01:41:48.326 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:41:48.327 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T01:41:48.351 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T01:41:48.351 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T01:41:48.351 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T01:41:48.352 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T01:41:48.354 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T01:41:48.357 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T01:41:48.359 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T01:41:48.434 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-04-26T01:41:48.448 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T01:41:48.448 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-04-26T01:41:48.448 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T01:41:48.448 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-04-26T01:41:48.448 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-26T01:41:48.449 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T01:41:48.450 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-26T01:41:48.451 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-26T01:41:48.452 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T01:41:48.453 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T01:41:48.454 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T01:41:48.455 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T01:41:48.456 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T01:41:48.457 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T01:41:48.458 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T01:41:48.459 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout:======================================================================================= 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 4 Packages 2024-04-26T01:41:48.460 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:41:48.461 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 222 M 2024-04-26T01:41:48.461 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-04-26T01:41:49.223 INFO:teuthology.orchestra.run.smithi037.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-26T01:41:49.531 INFO:teuthology.orchestra.run.smithi037.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.8 MB/s | 890 kB 00:00 2024-04-26T01:41:49.631 INFO:teuthology.orchestra.run.smithi037.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 1.4 MB/s | 145 kB 00:00 2024-04-26T01:41:49.987 INFO:teuthology.orchestra.run.smithi045.stdout:Transaction test succeeded. 2024-04-26T01:41:49.988 INFO:teuthology.orchestra.run.smithi045.stdout:Running transaction 2024-04-26T01:41:50.182 INFO:teuthology.orchestra.run.smithi037.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 4.3 MB/s | 5.4 MB 00:01 2024-04-26T01:41:50.257 INFO:teuthology.orchestra.run.smithi037.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 3.8 MB/s | 2.4 MB 00:00 2024-04-26T01:41:50.483 INFO:teuthology.orchestra.run.smithi037.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 5.0 MB/s | 1.5 MB 00:00 2024-04-26T01:41:51.033 INFO:teuthology.orchestra.run.smithi037.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 6.1 MB/s | 4.7 MB 00:00 2024-04-26T01:41:52.108 INFO:teuthology.orchestra.run.smithi037.stdout:(8/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x86 10 MB/s | 17 MB 00:01 2024-04-26T01:41:52.175 INFO:teuthology.orchestra.run.smithi037.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 384 kB/s | 25 kB 00:00 2024-04-26T01:41:52.359 INFO:teuthology.orchestra.run.smithi037.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 8.0 MB/s | 11 MB 00:01 2024-04-26T01:41:52.442 INFO:teuthology.orchestra.run.smithi037.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.7 MB/s | 225 kB 00:00 2024-04-26T01:41:52.580 INFO:teuthology.orchestra.run.smithi037.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 198 kB/s | 27 kB 00:00 2024-04-26T01:41:52.676 INFO:teuthology.orchestra.run.smithi037.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 333 kB/s | 32 kB 00:00 2024-04-26T01:41:52.876 INFO:teuthology.orchestra.run.smithi037.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.7 MB/s | 747 kB 00:00 2024-04-26T01:41:53.177 INFO:teuthology.orchestra.run.smithi037.stdout:(15/152): ceph-common-19.0.0-3156.gd69f13d8.el7 5.0 MB/s | 21 MB 00:04 2024-04-26T01:41:53.210 INFO:teuthology.orchestra.run.smithi037.stdout:(16/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 490 kB/s | 163 kB 00:00 2024-04-26T01:41:53.244 INFO:teuthology.orchestra.run.smithi037.stdout:(17/152): librados-devel-19.0.0-3156.gd69f13d8. 1.9 MB/s | 126 kB 00:00 2024-04-26T01:41:53.319 INFO:teuthology.orchestra.run.smithi037.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 4.5 MB/s | 500 kB 00:00 2024-04-26T01:41:53.386 INFO:teuthology.orchestra.run.smithi037.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 684 kB/s | 45 kB 00:00 2024-04-26T01:41:53.495 INFO:teuthology.orchestra.run.smithi037.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.2 MB/s | 135 kB 00:00 2024-04-26T01:41:53.514 INFO:teuthology.orchestra.run.smithi071.stdout: Preparing : 1/1 2024-04-26T01:41:53.620 INFO:teuthology.orchestra.run.smithi037.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.3 MB/s | 161 kB 00:00 2024-04-26T01:41:53.653 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:41:53.758 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:41:53.762 INFO:teuthology.orchestra.run.smithi037.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.2 MB/s | 320 kB 00:00 2024-04-26T01:41:53.811 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:41:53.854 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:41:53.879 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:41:53.896 INFO:teuthology.orchestra.run.smithi037.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.2 MB/s | 298 kB 00:00 2024-04-26T01:41:53.922 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:41:54.029 INFO:teuthology.orchestra.run.smithi037.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 751 kB/s | 99 kB 00:00 2024-04-26T01:41:54.196 INFO:teuthology.orchestra.run.smithi037.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 514 kB/s | 85 kB 00:00 2024-04-26T01:41:54.248 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:41:54.347 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:41:54.468 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:41:54.490 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:41:54.581 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:41:54.611 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:41:54.747 INFO:teuthology.orchestra.run.smithi037.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.6 MB/s | 5.4 MB 00:01 2024-04-26T01:41:54.763 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:41:54.808 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:41:54.836 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:41:54.855 INFO:teuthology.orchestra.run.smithi037.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 4.7 MB/s | 3.1 MB 00:00 2024-04-26T01:41:54.940 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:41:54.962 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:41:55.025 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:41:55.047 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:41:55.071 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:41:55.230 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:41:55.263 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:41:55.370 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:41:55.397 INFO:teuthology.orchestra.run.smithi037.stdout:(28/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 15 MB/s | 49 MB 00:03 2024-04-26T01:41:55.422 INFO:teuthology.orchestra.run.smithi037.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 60 kB/s | 34 kB 00:00 2024-04-26T01:41:55.448 INFO:teuthology.orchestra.run.smithi037.stdout:(30/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 244 kB/s | 170 kB 00:00 2024-04-26T01:41:55.476 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:41:55.481 INFO:teuthology.orchestra.run.smithi037.stdout:(31/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.7 MB/s | 142 kB 00:00 2024-04-26T01:41:55.512 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:41:55.547 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:41:55.582 INFO:teuthology.orchestra.run.smithi037.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-26T01:41:55.613 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:41:55.647 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:41:55.648 INFO:teuthology.orchestra.run.smithi037.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 745 kB/s | 49 kB 00:00 2024-04-26T01:41:55.699 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:41:55.724 INFO:teuthology.orchestra.run.smithi037.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 222 kB/s | 16 kB 00:00 2024-04-26T01:41:55.731 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:41:55.759 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:41:55.807 INFO:teuthology.orchestra.run.smithi037.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 273 kB 00:00 2024-04-26T01:41:55.830 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:41:55.883 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:41:55.913 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:41:55.933 INFO:teuthology.orchestra.run.smithi037.stdout:(36/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 6.9 MB/s | 3.5 MB 00:00 2024-04-26T01:41:55.957 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:41:55.983 INFO:teuthology.orchestra.run.smithi037.stdout:(37/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 4.3 MB/s | 762 kB 00:00 2024-04-26T01:41:56.000 INFO:teuthology.orchestra.run.smithi037.stdout:(38/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 276 kB/s | 18 kB 00:00 2024-04-26T01:41:56.034 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:41:56.066 INFO:teuthology.orchestra.run.smithi045.stdout: Preparing : 1/1 2024-04-26T01:41:56.067 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:41:56.119 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:41:56.148 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:41:56.162 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:41:56.200 INFO:teuthology.orchestra.run.smithi037.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 372 kB/s | 80 kB 00:00 2024-04-26T01:41:56.201 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:41:56.217 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:41:56.255 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:41:56.259 INFO:teuthology.orchestra.run.smithi037.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-04-26T01:41:56.262 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:41:56.284 INFO:teuthology.orchestra.run.smithi037.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 195 kB 00:00 2024-04-26T01:41:56.297 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:41:56.322 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:41:56.326 INFO:teuthology.orchestra.run.smithi037.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.8 MB/s | 253 kB 00:00 2024-04-26T01:41:56.337 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:41:56.365 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:41:56.420 INFO:teuthology.orchestra.run.smithi037.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 9.3 MB/s | 1.2 MB 00:00 2024-04-26T01:41:56.452 INFO:teuthology.orchestra.run.smithi037.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 905 kB/s | 106 kB 00:00 2024-04-26T01:41:56.472 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:41:56.494 INFO:teuthology.orchestra.run.smithi037.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 3.3 MB/s | 135 kB 00:00 2024-04-26T01:41:56.519 INFO:teuthology.orchestra.run.smithi037.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.9 MB/s | 125 kB 00:00 2024-04-26T01:41:56.531 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:41:56.553 INFO:teuthology.orchestra.run.smithi037.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 215 kB 00:00 2024-04-26T01:41:56.594 INFO:teuthology.orchestra.run.smithi037.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 9.3 MB/s | 689 kB 00:00 2024-04-26T01:41:56.645 INFO:teuthology.orchestra.run.smithi037.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 11 MB/s | 943 kB 00:00 2024-04-26T01:41:56.670 INFO:teuthology.orchestra.run.smithi037.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-26T01:41:56.695 INFO:teuthology.orchestra.run.smithi037.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 445 kB/s | 21 kB 00:00 2024-04-26T01:41:56.729 INFO:teuthology.orchestra.run.smithi037.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-04-26T01:41:56.754 INFO:teuthology.orchestra.run.smithi037.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 598 kB/s | 33 kB 00:00 2024-04-26T01:41:56.763 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:41:56.822 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:41:56.855 INFO:teuthology.orchestra.run.smithi037.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-26T01:41:56.880 INFO:teuthology.orchestra.run.smithi037.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-04-26T01:41:56.905 INFO:teuthology.orchestra.run.smithi037.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 227 kB/s | 11 kB 00:00 2024-04-26T01:41:56.939 INFO:teuthology.orchestra.run.smithi037.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T01:41:56.950 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:41:56.964 INFO:teuthology.orchestra.run.smithi037.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 805 kB/s | 45 kB 00:00 2024-04-26T01:41:56.973 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:41:57.014 INFO:teuthology.orchestra.run.smithi037.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-26T01:41:57.057 INFO:teuthology.orchestra.run.smithi037.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 6.7 MB/s | 582 kB 00:00 2024-04-26T01:41:57.065 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:41:57.065 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:41:57.082 INFO:teuthology.orchestra.run.smithi037.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 417 kB/s | 25 kB 00:00 2024-04-26T01:41:57.102 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:41:57.107 INFO:teuthology.orchestra.run.smithi037.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 967 kB/s | 47 kB 00:00 2024-04-26T01:41:57.162 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:41:57.166 INFO:teuthology.orchestra.run.smithi037.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.5 MB/s | 292 kB 00:00 2024-04-26T01:41:57.199 INFO:teuthology.orchestra.run.smithi037.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.1 MB/s | 188 kB 00:00 2024-04-26T01:41:57.223 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:41:57.225 INFO:teuthology.orchestra.run.smithi037.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 629 kB/s | 35 kB 00:00 2024-04-26T01:41:57.259 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:41:57.266 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:41:57.304 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:41:57.308 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:41:57.335 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:41:57.346 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:41:57.350 INFO:teuthology.orchestra.run.smithi037.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-04-26T01:41:57.441 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:41:57.450 INFO:teuthology.orchestra.run.smithi037.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-04-26T01:41:57.470 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:41:57.501 INFO:teuthology.orchestra.run.smithi037.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-26T01:41:57.506 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:41:57.516 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:41:57.537 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:41:57.559 INFO:teuthology.orchestra.run.smithi037.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 8.8 MB/s | 911 kB 00:00 2024-04-26T01:41:57.562 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:41:57.566 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:41:57.584 INFO:teuthology.orchestra.run.smithi037.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-26T01:41:57.621 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:41:57.694 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:41:57.713 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:41:57.725 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:41:57.745 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:41:57.762 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:41:57.768 INFO:teuthology.orchestra.run.smithi037.stdout:(71/152): postgresql-server-13.14-1.el9.x86_64. 28 MB/s | 5.8 MB 00:00 2024-04-26T01:41:57.808 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:41:57.819 INFO:teuthology.orchestra.run.smithi037.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 4.6 MB/s | 1.0 MB 00:00 2024-04-26T01:41:57.844 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:41:57.844 INFO:teuthology.orchestra.run.smithi037.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T01:41:57.857 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:41:57.889 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:41:57.930 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:41:57.952 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:41:57.986 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:41:58.003 INFO:teuthology.orchestra.run.smithi037.stdout:(74/152): ceph-mgr-diskprediction-local-19.0.0- 2.9 MB/s | 7.4 MB 00:02 2024-04-26T01:41:58.030 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:41:58.068 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:41:58.106 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:41:58.118 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:41:58.128 INFO:teuthology.orchestra.run.smithi037.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 6.0 MB 00:00 2024-04-26T01:41:58.145 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:41:58.149 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:41:58.153 INFO:teuthology.orchestra.run.smithi037.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 835 kB/s | 244 kB 00:00 2024-04-26T01:41:58.187 INFO:teuthology.orchestra.run.smithi037.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.4 MB/s | 248 kB 00:00 2024-04-26T01:41:58.189 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:41:58.191 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:41:58.222 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:41:58.229 INFO:teuthology.orchestra.run.smithi037.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 357 kB/s | 26 kB 00:00 2024-04-26T01:41:58.249 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:41:58.282 INFO:teuthology.orchestra.run.smithi037.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 153 kB/s | 19 kB 00:00 2024-04-26T01:41:58.286 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:41:58.315 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:41:58.321 INFO:teuthology.orchestra.run.smithi037.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-04-26T01:41:58.321 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:41:58.356 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:41:58.363 INFO:teuthology.orchestra.run.smithi037.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 447 kB/s | 35 kB 00:00 2024-04-26T01:41:58.373 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:41:58.381 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:41:58.402 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:41:58.406 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:41:58.461 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:41:58.482 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:41:58.496 INFO:teuthology.orchestra.run.smithi037.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-26T01:41:58.529 INFO:teuthology.orchestra.run.smithi037.stdout:(83/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 250 kB 00:00 2024-04-26T01:41:58.540 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:41:58.555 INFO:teuthology.orchestra.run.smithi037.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.2 MB/s | 222 kB 00:00 2024-04-26T01:41:58.572 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:41:58.597 INFO:teuthology.orchestra.run.smithi037.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 1.9 MB/s | 77 kB 00:00 2024-04-26T01:41:58.616 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:41:58.618 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:41:58.622 INFO:teuthology.orchestra.run.smithi037.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 648 kB/s | 42 kB 00:00 2024-04-26T01:41:58.645 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:41:58.664 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:41:58.672 INFO:teuthology.orchestra.run.smithi037.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-26T01:41:58.692 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:41:58.706 INFO:teuthology.orchestra.run.smithi037.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-26T01:41:58.717 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:41:58.726 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:41:58.748 INFO:teuthology.orchestra.run.smithi037.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.7 MB/s | 279 kB 00:00 2024-04-26T01:41:58.760 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:41:58.765 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:41:58.782 INFO:teuthology.orchestra.run.smithi037.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 480 kB/s | 51 kB 00:00 2024-04-26T01:41:58.797 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:41:58.807 INFO:teuthology.orchestra.run.smithi037.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 555 kB/s | 54 kB 00:00 2024-04-26T01:41:58.827 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:41:58.838 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:41:58.861 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:41:58.874 INFO:teuthology.orchestra.run.smithi037.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 463 kB/s | 42 kB 00:00 2024-04-26T01:41:58.895 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:41:58.966 INFO:teuthology.orchestra.run.smithi037.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 311 kB/s | 44 kB 00:00 2024-04-26T01:41:58.981 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:41:59.012 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:41:59.038 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:41:59.155 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:41:59.201 INFO:teuthology.orchestra.run.smithi037.stdout:(94/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 39 MB/s | 16 MB 00:00 2024-04-26T01:41:59.224 INFO:teuthology.orchestra.run.smithi037.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 661 kB/s | 221 kB 00:00 2024-04-26T01:41:59.239 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:41:59.250 INFO:teuthology.orchestra.run.smithi037.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 132 kB/s | 36 kB 00:00 2024-04-26T01:41:59.286 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:41:59.292 INFO:teuthology.orchestra.run.smithi037.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.2 MB/s | 47 kB 00:00 2024-04-26T01:41:59.316 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:41:59.317 INFO:teuthology.orchestra.run.smithi037.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T01:41:59.342 INFO:teuthology.orchestra.run.smithi037.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 639 kB/s | 58 kB 00:00 2024-04-26T01:41:59.364 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:41:59.409 INFO:teuthology.orchestra.run.smithi037.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-04-26T01:41:59.443 INFO:teuthology.orchestra.run.smithi037.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 646 kB/s | 64 kB 00:00 2024-04-26T01:41:59.468 INFO:teuthology.orchestra.run.smithi037.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 363 kB/s | 45 kB 00:00 2024-04-26T01:41:59.474 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:41:59.510 INFO:teuthology.orchestra.run.smithi037.stdout:(103/152): lua-devel-5.4.4-4.el9.x86_64.rpm 552 kB/s | 22 kB 00:00 2024-04-26T01:41:59.561 INFO:teuthology.orchestra.run.smithi037.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 9.3 MB/s | 863 kB 00:00 2024-04-26T01:41:59.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:41:59.693 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:41:59.710 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:41:59.727 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:41:59.770 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:41:59.802 INFO:teuthology.orchestra.run.smithi037.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 81 kB/s | 19 kB 00:00 2024-04-26T01:41:59.808 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:41:59.850 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:41:59.875 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://mirror.arizona.edu/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 128.196.204.204) 2024-04-26T01:41:59.953 INFO:teuthology.orchestra.run.smithi037.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 710 kB/s | 309 kB 00:00 2024-04-26T01:42:00.025 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:42:00.061 INFO:teuthology.orchestra.run.smithi037.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 928 kB/s | 548 kB 00:00 2024-04-26T01:42:00.077 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:42:00.095 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://mirror.arizona.edu/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 128.196.204.204) 2024-04-26T01:42:00.134 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:42:00.153 INFO:teuthology.orchestra.run.smithi037.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 532 kB/s | 49 kB 00:00 2024-04-26T01:42:00.176 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://mirror.arizona.edu/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 128.196.204.204) 2024-04-26T01:42:00.177 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://mirror.arizona.edu/fedora-epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 128.196.204.204) 2024-04-26T01:42:00.207 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:42:00.245 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:42:00.253 INFO:teuthology.orchestra.run.smithi037.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 677 kB/s | 67 kB 00:00 2024-04-26T01:42:00.282 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:42:00.337 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:42:00.372 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:42:00.416 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:42:00.432 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 23.144.216.11) 2024-04-26T01:42:00.447 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:42:00.457 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:42:00.490 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:42:00.512 INFO:teuthology.orchestra.run.smithi037.stdout:(110/152): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-04-26T01:42:00.514 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 23.144.216.11) 2024-04-26T01:42:00.532 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:42:00.564 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-9.0.0-10.el9.x86_64.rpm (IP: 23.144.216.11) 2024-04-26T01:42:00.602 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:42:00.632 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:42:00.646 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:42:00.676 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:42:00.677 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://mirror.arizona.edu/fedora-epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 128.196.204.204) 2024-04-26T01:42:00.679 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] libarrow-doc-9.0.0-10.el9.noarch.rpm: Status code: 404 for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/l/libarrow-doc-9.0.0-10.el9.noarch.rpm (IP: 23.144.216.11) 2024-04-26T01:42:00.690 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:42:00.717 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:42:00.779 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:42:00.785 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://mirror.arizona.edu/fedora-epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 128.196.204.204) 2024-04-26T01:42:00.816 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:42:00.818 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:42:00.835 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for https://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 23.144.216.11) 2024-04-26T01:42:00.851 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:42:00.882 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:42:00.885 INFO:teuthology.orchestra.run.smithi037.stdout:[MIRROR] parquet-libs-9.0.0-10.el9.x86_64.rpm: Status code: 404 for http://ix-denver.mm.fcix.net/epel/9/Everything/x86_64/Packages/p/parquet-libs-9.0.0-10.el9.x86_64.rpm (IP: 23.144.216.11) 2024-04-26T01:42:00.907 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:42:00.921 INFO:teuthology.orchestra.run.smithi037.stdout:(111/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 29 kB/s | 26 kB 00:00 2024-04-26T01:42:00.932 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:42:00.994 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:42:01.010 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:42:01.024 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:42:01.101 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:42:01.148 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:42:01.171 INFO:teuthology.orchestra.run.smithi037.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-04-26T01:42:01.181 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:42:01.209 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:42:01.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:42:01.283 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:42:01.288 INFO:teuthology.orchestra.run.smithi037.stdout:(113/152): libarrow-9.0.0-10.el9.x86_64.rpm 3.0 MB/s | 4.4 MB 00:01 2024-04-26T01:42:01.313 INFO:teuthology.orchestra.run.smithi037.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 307 kB/s | 43 kB 00:00 2024-04-26T01:42:01.333 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:42:01.360 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:42:01.395 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:42:01.428 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:42:01.447 INFO:teuthology.orchestra.run.smithi037.stdout:(115/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 898 kB/s | 838 kB 00:00 2024-04-26T01:42:01.470 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:42:01.472 INFO:teuthology.orchestra.run.smithi037.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 95 kB/s | 15 kB 00:00 2024-04-26T01:42:01.497 INFO:teuthology.orchestra.run.smithi037.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 155 kB/s | 32 kB 00:00 2024-04-26T01:42:01.504 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:42:01.564 INFO:teuthology.orchestra.run.smithi037.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-26T01:42:01.593 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:42:01.644 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:42:01.674 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:42:01.681 INFO:teuthology.orchestra.run.smithi037.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 304 kB/s | 35 kB 00:00 2024-04-26T01:42:01.722 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:42:01.750 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:42:01.784 INFO:teuthology.orchestra.run.smithi037.stdout:(120/152): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-04-26T01:42:01.833 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:42:01.875 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:42:01.935 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:42:01.984 INFO:teuthology.orchestra.run.smithi037.stdout:(121/152): python3-cherrypy-18.6.1-2.el9.noarch 702 kB/s | 358 kB 00:00 2024-04-26T01:42:02.051 INFO:teuthology.orchestra.run.smithi037.stdout:(122/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 548 kB/s | 303 kB 00:00 2024-04-26T01:42:02.143 INFO:teuthology.orchestra.run.smithi037.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 5.7 MB/s | 2.0 MB 00:00 2024-04-26T01:42:02.168 INFO:teuthology.orchestra.run.smithi037.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-26T01:42:02.193 INFO:teuthology.orchestra.run.smithi037.stdout:(125/152): python3-grpcio-tools-1.46.7-10.el9.x 693 kB/s | 144 kB 00:00 2024-04-26T01:42:02.244 INFO:teuthology.orchestra.run.smithi037.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-26T01:42:02.270 INFO:teuthology.orchestra.run.smithi037.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 232 kB/s | 23 kB 00:00 2024-04-26T01:42:02.294 INFO:teuthology.orchestra.run.smithi037.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-26T01:42:02.353 INFO:teuthology.orchestra.run.smithi037.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-04-26T01:42:02.403 INFO:teuthology.orchestra.run.smithi037.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 428 kB/s | 46 kB 00:00 2024-04-26T01:42:02.470 INFO:teuthology.orchestra.run.smithi037.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 680 kB/s | 79 kB 00:00 2024-04-26T01:42:02.512 INFO:teuthology.orchestra.run.smithi037.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 534 kB/s | 58 kB 00:00 2024-04-26T01:42:02.596 INFO:teuthology.orchestra.run.smithi037.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-04-26T01:42:02.612 INFO:teuthology.orchestra.run.smithi037.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-26T01:42:02.696 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:42:02.730 INFO:teuthology.orchestra.run.smithi037.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 686 kB/s | 90 kB 00:00 2024-04-26T01:42:02.746 INFO:teuthology.orchestra.run.smithi037.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 213 kB/s | 28 kB 00:00 2024-04-26T01:42:02.747 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:42:02.772 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:42:02.838 INFO:teuthology.orchestra.run.smithi037.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 286 kB/s | 31 kB 00:00 2024-04-26T01:42:02.875 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:42:02.931 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:42:02.947 INFO:teuthology.orchestra.run.smithi037.stdout:(138/152): python3-rsa-4.9-2.el9.noarch.rpm 550 kB/s | 59 kB 00:00 2024-04-26T01:42:03.023 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:42:03.031 INFO:teuthology.orchestra.run.smithi037.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 666 kB/s | 188 kB 00:00 2024-04-26T01:42:03.057 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:42:03.093 INFO:teuthology.orchestra.run.smithi037.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 246 kB/s | 36 kB 00:00 2024-04-26T01:42:03.118 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:42:03.139 INFO:teuthology.orchestra.run.smithi037.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 472 kB/s | 51 kB 00:00 2024-04-26T01:42:03.198 INFO:teuthology.orchestra.run.smithi037.stdout:(142/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-26T01:42:03.205 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:42:03.224 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:42:03.232 INFO:teuthology.orchestra.run.smithi037.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 983 kB/s | 90 kB 00:00 2024-04-26T01:42:03.241 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:42:03.250 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:42:03.271 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:42:03.307 INFO:teuthology.orchestra.run.smithi037.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-26T01:42:03.332 INFO:teuthology.orchestra.run.smithi037.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-26T01:42:03.352 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:42:03.416 INFO:teuthology.orchestra.run.smithi037.stdout:(146/152): python3-kubernetes-26.1.0-2.el9.noar 912 kB/s | 1.0 MB 00:01 2024-04-26T01:42:03.466 INFO:teuthology.orchestra.run.smithi037.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.2 MB/s | 192 kB 00:00 2024-04-26T01:42:03.559 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:42:03.596 INFO:teuthology.orchestra.run.smithi071.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:42:03.685 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:42:03.717 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:42:03.753 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:42:04.000 INFO:teuthology.orchestra.run.smithi037.stdout:(148/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.8 MB/s | 3.4 MB 00:00 2024-04-26T01:42:04.042 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:42:04.151 INFO:teuthology.orchestra.run.smithi037.stdout:(149/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 5.1 MB/s | 3.1 MB 00:00 2024-04-26T01:42:04.156 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:42:04.232 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:42:04.258 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:42:04.293 INFO:teuthology.orchestra.run.smithi037.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.7 MB/s | 1.3 MB 00:00 2024-04-26T01:42:04.329 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:42:04.351 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:42:04.376 INFO:teuthology.orchestra.run.smithi037.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-26T01:42:04.426 INFO:teuthology.orchestra.run.smithi037.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 1.4 MB/s | 1.6 MB 00:01 2024-04-26T01:42:04.437 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:42:04.437 INFO:teuthology.orchestra.run.smithi037.stdout:Total 14 MB/s | 222 MB 00:15 2024-04-26T01:42:05.451 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:42:05.524 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:42:05.552 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:42:05.585 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:42:05.914 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:42:05.956 INFO:teuthology.orchestra.run.smithi045.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:42:06.047 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:42:06.077 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:42:06.114 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:42:06.621 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:42:06.625 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:06.650 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:42:06.673 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:06.722 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:42:06.752 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:42:07.217 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:07.269 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:42:07.284 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-04-26T01:42:07.361 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:42:07.362 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:42:07.362 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:42:07.362 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:07.395 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:42:07.539 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-04-26T01:42:07.539 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-04-26T01:42:09.144 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:09.183 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:09.755 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:09.837 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:42:09.920 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:42:09.920 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:42:09.920 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:42:09.920 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:09.954 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:42:10.234 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-04-26T01:42:10.234 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-04-26T01:42:17.020 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-04-26T01:42:17.126 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T01:42:17.220 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T01:42:17.358 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T01:42:17.423 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T01:42:17.460 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T01:42:17.603 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T01:42:18.073 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T01:42:18.191 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:42:18.380 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:42:18.417 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:42:18.558 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:42:18.601 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /sys 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /proc 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /mnt 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /var/tmp 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /home 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /root 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout:skipping the directory /tmp 2024-04-26T01:42:18.821 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:18.970 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T01:42:19.011 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:42:19.026 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:42:19.027 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:19.027 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:42:19.027 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:19.038 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T01:42:19.090 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:42:19.265 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:42:19.289 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:42:19.369 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-26T01:42:19.407 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:42:19.488 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T01:42:19.814 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:42:19.876 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T01:42:20.079 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T01:42:20.222 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-26T01:42:20.266 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T01:42:20.317 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T01:42:20.425 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T01:42:20.470 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T01:42:20.514 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:42:20.534 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T01:42:20.539 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:42:20.539 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:20.539 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:42:20.539 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:42:20.540 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:42:20.540 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:20.609 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T01:42:20.648 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T01:42:20.765 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:42:20.775 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T01:42:20.787 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:42:20.787 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:20.787 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:42:20.787 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:42:20.788 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:42:20.788 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:20.852 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T01:42:20.892 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T01:42:20.981 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T01:42:21.101 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T01:42:21.137 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-26T01:42:21.181 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-26T01:42:21.209 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-26T01:42:21.232 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:42:21.254 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:42:21.254 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:21.254 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:42:21.254 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:42:21.254 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:42:21.255 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:21.301 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T01:42:21.362 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T01:42:21.363 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:42:21.420 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:42:21.462 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T01:42:21.494 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:42:21.650 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T01:42:21.679 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:42:21.726 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T01:42:21.734 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /sys 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /proc 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /mnt 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /var/tmp 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /home 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /root 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout:skipping the directory /tmp 2024-04-26T01:42:21.793 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:21.812 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:42:21.902 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:42:21.963 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:42:21.979 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:42:21.980 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:21.980 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:42:21.980 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:22.052 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:42:22.101 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:42:22.133 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:42:22.161 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:42:22.440 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T01:42:22.554 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:42:22.585 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:42:22.617 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:42:22.678 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:42:22.717 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:42:22.750 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T01:42:22.800 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T01:42:22.850 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T01:42:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-26T01:42:22.975 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T01:42:23.328 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T01:42:23.388 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T01:42:23.473 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T01:42:23.516 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:42:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:42:23.543 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:23.543 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:42:23.543 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:42:23.543 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:42:23.543 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:23.600 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T01:42:23.645 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T01:42:23.686 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T01:42:23.754 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:42:23.757 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T01:42:23.775 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:42:23.785 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:42:23.798 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:42:23.799 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:23.799 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:42:23.799 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:42:23.799 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:42:23.799 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:23.801 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T01:42:23.863 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T01:42:23.932 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T01:42:24.145 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T01:42:24.215 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T01:42:24.276 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:42:24.277 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T01:42:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:42:24.300 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:24.300 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:42:24.300 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:42:24.300 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:42:24.300 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:24.338 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T01:42:24.406 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:42:24.455 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:42:24.483 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T01:42:24.511 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:42:24.526 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T01:42:24.575 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T01:42:24.633 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T01:42:24.666 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-26T01:42:24.706 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:42:24.749 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:42:24.751 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:42:24.777 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:42:24.832 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:42:24.860 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T01:42:24.880 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:42:24.921 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:42:25.044 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T01:42:25.048 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:42:25.063 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:42:25.071 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:42:25.089 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T01:42:25.094 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:42:25.094 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:25.094 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:42:25.094 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:42:25.094 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:42:25.094 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:25.101 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:42:25.128 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T01:42:25.133 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:42:25.138 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:42:25.149 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:42:25.152 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:42:25.175 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T01:42:25.235 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T01:42:25.246 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:42:25.293 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T01:42:25.295 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:42:25.329 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T01:42:25.397 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T01:42:25.449 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T01:42:25.525 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T01:42:25.570 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T01:42:25.614 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:42:25.642 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:42:25.675 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:42:25.720 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T01:42:25.737 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:42:25.765 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:42:25.793 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T01:42:25.862 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T01:42:25.938 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T01:42:26.100 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T01:42:26.168 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T01:42:26.221 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:42:26.258 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:42:26.281 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:42:26.281 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:26.281 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:42:26.281 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:42:26.282 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:42:26.282 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:26.326 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:42:26.347 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:42:26.347 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:26.347 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:42:26.348 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:26.392 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:42:26.415 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:42:26.415 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:26.415 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:42:26.415 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:42:26.415 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:42:26.416 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:26.718 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:42:26.741 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:42:26.741 INFO:teuthology.orchestra.run.smithi071.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:26.742 INFO:teuthology.orchestra.run.smithi071.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:42:26.742 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:42:26.742 INFO:teuthology.orchestra.run.smithi071.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:42:26.742 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:26.857 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:42:26.891 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:42:27.191 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T01:42:27.249 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T01:42:27.282 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T01:42:27.413 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T01:42:27.479 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T01:42:27.590 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T01:42:27.634 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T01:42:27.881 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T01:42:27.908 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:42:27.936 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T01:42:27.975 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:42:28.064 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T01:42:28.081 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:42:28.254 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:42:28.293 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:42:28.318 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:42:28.318 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:28.318 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:42:28.319 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:42:28.319 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:42:28.319 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:28.377 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:42:28.389 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:42:28.483 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:42:28.535 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:42:29.002 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T01:42:29.167 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T01:42:29.512 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:42:29.538 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:42:29.562 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:42:29.562 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:29.562 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:42:29.562 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:42:29.563 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:42:29.563 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:29.606 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:42:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:42:29.629 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:29.629 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:42:29.629 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:29.690 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:42:29.713 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:42:29.713 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:29.714 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:42:29.714 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:42:29.714 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:42:29.714 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:30.032 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:42:30.055 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:42:30.055 INFO:teuthology.orchestra.run.smithi045.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:30.056 INFO:teuthology.orchestra.run.smithi045.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:42:30.056 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:42:30.056 INFO:teuthology.orchestra.run.smithi045.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:42:30.056 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:30.874 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T01:42:30.979 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T01:42:31.031 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T01:42:31.086 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T01:42:31.314 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:42:31.355 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:42:31.405 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:42:31.437 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:42:31.464 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:42:31.488 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T01:42:31.533 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:42:31.650 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:42:31.681 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-26T01:42:31.702 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:42:31.723 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-26T01:42:31.740 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:42:31.773 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T01:42:32.334 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:42:32.340 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-26T01:42:32.372 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:42:32.386 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:42:32.392 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:42:32.487 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-26T01:42:32.512 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-26T01:42:33.216 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:42:33.304 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:42:33.361 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:42:33.444 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:42:33.474 INFO:teuthology.orchestra.run.smithi071.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:42:33.475 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:42:33.559 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:42:33.592 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:42:33.592 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:42:33.677 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:42:33.750 INFO:teuthology.orchestra.run.smithi071.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:42:34.910 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:42:34.952 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:42:34.993 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:42:35.024 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:42:35.051 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:42:35.244 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:42:35.289 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:42:35.319 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:42:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:35.956 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:42:35.990 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:42:36.011 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:42:36.306 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-26T01:42:36.354 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:42:36.565 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-26T01:42:36.565 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T01:42:36.566 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T01:42:36.566 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:36.614 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:42:36.898 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:42:36.989 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:42:37.022 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:42:37.106 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:42:37.176 INFO:teuthology.orchestra.run.smithi045.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:42:37.176 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:42:37.260 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:42:37.293 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:42:37.293 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:42:37.384 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:42:37.452 INFO:teuthology.orchestra.run.smithi045.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:42:39.649 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:42:39.650 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:42:39.651 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:42:39.652 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:42:39.653 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:42:39.654 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:42:39.655 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:42:39.656 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:42:39.657 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:42:39.658 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:42:39.659 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:42:39.660 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:42:41.713 INFO:teuthology.orchestra.run.smithi071.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout:Upgraded: 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout:Installed: 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:42:41.714 INFO:teuthology.orchestra.run.smithi071.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.715 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:42:41.716 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:42:41.717 INFO:teuthology.orchestra.run.smithi071.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:42:41.718 INFO:teuthology.orchestra.run.smithi071.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:42:41.719 INFO:teuthology.orchestra.run.smithi071.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:42:41.720 INFO:teuthology.orchestra.run.smithi071.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:42:41.721 INFO:teuthology.orchestra.run.smithi071.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:42:41.722 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:42:41.723 INFO:teuthology.orchestra.run.smithi071.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:41.724 INFO:teuthology.orchestra.run.smithi071.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:42:41.725 INFO:teuthology.orchestra.run.smithi071.stdout:Complete! 2024-04-26T01:42:42.111 DEBUG:teuthology.parallel:result is None 2024-04-26T01:42:43.193 INFO:teuthology.orchestra.run.smithi045.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:42:43.193 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:42:43.193 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:42:43.194 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:42:43.195 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:42:43.196 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:42:43.197 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:42:43.198 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:42:43.199 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:42:43.200 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:42:43.201 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:42:43.202 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:42:43.203 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:42:43.204 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:42:43.204 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:42:43.204 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:42:43.204 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:42:43.204 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:42:45.281 INFO:teuthology.orchestra.run.smithi045.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout:Upgraded: 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout:Installed: 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:42:45.282 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.283 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:42:45.284 INFO:teuthology.orchestra.run.smithi045.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:42:45.285 INFO:teuthology.orchestra.run.smithi045.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:42:45.286 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:42:45.287 INFO:teuthology.orchestra.run.smithi045.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:42:45.288 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:42:45.289 INFO:teuthology.orchestra.run.smithi045.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:42:45.290 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:42:45.291 INFO:teuthology.orchestra.run.smithi045.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:42:45.292 INFO:teuthology.orchestra.run.smithi045.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:42:45.293 INFO:teuthology.orchestra.run.smithi045.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:42:45.293 INFO:teuthology.orchestra.run.smithi045.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:42:45.293 INFO:teuthology.orchestra.run.smithi045.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:42:45.293 INFO:teuthology.orchestra.run.smithi045.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:42:45.293 INFO:teuthology.orchestra.run.smithi045.stdout: 2024-04-26T01:42:45.293 INFO:teuthology.orchestra.run.smithi045.stdout:Complete! 2024-04-26T01:42:45.661 DEBUG:teuthology.parallel:result is None 2024-04-26T01:42:49.848 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-04-26T01:42:49.849 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:50.086 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:42:50.141 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-26T01:42:50.141 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:50.141 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T01:42:50.141 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:51.827 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:42:51.912 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-26T01:42:51.913 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:51.913 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T01:42:51.913 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:42:51.913 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T01:42:51.913 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:52.178 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:42:52.250 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-26T01:42:52.250 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:52.250 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T01:42:52.250 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:42:52.250 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T01:42:52.250 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:52.795 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:42:52.879 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-26T01:42:52.880 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:52.880 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T01:42:52.880 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:42:52.880 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T01:42:52.880 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:53.030 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T01:42:53.098 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T01:42:53.180 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T01:42:53.454 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T01:42:53.518 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T01:42:53.610 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T01:42:53.731 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:42:53.931 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:42:54.009 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T01:42:54.059 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T01:42:54.090 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T01:42:54.697 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:42:54.771 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-26T01:42:54.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-26T01:42:54.961 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-26T01:42:55.174 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:42:56.546 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-26T01:42:56.697 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:42:57.875 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-26T01:42:57.905 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:42:58.060 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-26T01:42:58.245 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-26T01:42:58.282 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:42:58.356 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-26T01:42:58.356 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:58.356 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T01:42:58.356 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:42:58.356 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T01:42:58.356 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:58.416 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:42:58.456 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-26T01:42:58.604 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-26T01:42:58.685 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-26T01:42:59.749 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T01:42:59.777 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:42:59.850 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-26T01:42:59.851 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:42:59.851 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T01:42:59.851 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:42:59.851 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T01:42:59.851 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:42:59.911 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:43:00.007 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-26T01:43:00.007 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:43:00.008 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T01:43:00.008 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:43:00.087 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:43:00.197 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-26T01:43:00.197 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:43:00.197 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T01:43:00.197 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:43:00.197 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T01:43:00.198 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:43:00.604 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:43:00.707 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-26T01:43:00.707 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T01:43:00.707 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T01:43:00.708 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:43:00.708 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T01:43:00.708 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:43:06.047 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-26T01:43:06.105 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-26T01:43:06.200 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-26T01:43:06.249 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T01:43:06.332 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T01:43:06.697 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-26T01:43:06.757 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T01:43:06.784 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-26T01:43:07.666 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:43:07.725 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:43:07.783 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T01:43:09.205 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-26T01:43:09.308 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T01:43:09.378 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T01:43:09.576 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T01:43:09.649 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T01:43:09.649 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:43:09.784 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T01:43:09.831 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:43:09.831 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:43:10.005 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T01:43:10.140 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:43:10.438 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:43:10.439 INFO:teuthology.orchestra.run.smithi037.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-26T01:43:10.439 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:43:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-26T01:43:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-26T01:43:16.406 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-26T01:43:16.407 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-26T01:43:16.408 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-26T01:43:16.421 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T01:43:16.422 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T01:43:16.423 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T01:43:16.424 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T01:43:16.425 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T01:43:16.426 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T01:43:16.427 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T01:43:16.428 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T01:43:16.429 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T01:43:16.430 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T01:43:18.873 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.874 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.875 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.876 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T01:43:18.877 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T01:43:18.878 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T01:43:18.879 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T01:43:18.880 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T01:43:18.881 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.882 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T01:43:18.883 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T01:43:18.884 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T01:43:18.885 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T01:43:18.885 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:43:18.885 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-04-26T01:43:19.490 DEBUG:teuthology.parallel:result is None 2024-04-26T01:43:19.490 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:43:19.807 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:43:19.866 INFO:teuthology.orchestra.run.smithi037.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:43:19.867 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:43:19.867 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:43:19.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:43:20.176 DEBUG:teuthology.orchestra.run.smithi045:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:43:20.202 INFO:teuthology.orchestra.run.smithi045.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:43:20.202 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:43:20.202 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:43:20.204 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:43:20.346 DEBUG:teuthology.orchestra.run.smithi071:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T01:43:20.371 INFO:teuthology.orchestra.run.smithi071.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-26T01:43:20.371 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-26T01:43:20.371 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-26T01:43:20.373 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T01:43:20.373 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:43:20.373 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:43:20.447 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:43:20.447 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:43:20.479 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:43:20.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T01:43:20.509 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T01:43:20.509 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:43:20.509 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:43:20.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:43:20.685 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:43:20.685 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:43:20.714 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:43:20.782 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:43:20.783 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T01:43:20.806 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T01:43:20.871 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T01:43:20.871 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:43:20.871 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:43:20.943 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:43:21.058 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:43:21.058 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:43:21.083 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:43:21.149 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:43:21.149 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T01:43:21.173 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T01:43:21.237 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T01:43:21.237 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:43:21.238 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:43:21.305 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:43:21.423 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:43:21.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:43:21.448 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:43:21.513 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:43:21.513 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T01:43:21.536 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T01:43:21.599 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T01:43:21.694 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T01:43:21.694 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:43:21.695 INFO:tasks.cephadm:Cluster fsid is 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:43:21.695 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T01:43:21.695 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.45', 'mon.c': '172.21.15.71'} 2024-04-26T01:43:21.695 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-04-26T01:43:21.696 INFO:tasks.cephadm:First mgr is x 2024-04-26T01:43:21.696 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T01:43:21.696 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-04-26T01:43:21.763 DEBUG:teuthology.orchestra.run.smithi045:> sudo hostname $(hostname -s) 2024-04-26T01:43:21.787 DEBUG:teuthology.orchestra.run.smithi071:> sudo hostname $(hostname -s) 2024-04-26T01:43:21.811 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T01:43:21.811 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:43:21.958 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T01:43:22.109 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:43:22.110 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-26T01:43:22.110 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:22.623 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:43 /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:22.623 DEBUG:teuthology.orchestra.run.smithi045:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.070 INFO:teuthology.orchestra.run.smithi045.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:43 /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.070 DEBUG:teuthology.orchestra.run.smithi071:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.595 INFO:teuthology.orchestra.run.smithi071.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 26 01:43 /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.595 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.645 DEBUG:teuthology.orchestra.run.smithi045:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.664 DEBUG:teuthology.orchestra.run.smithi071:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T01:43:23.699 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-26T01:43:23.699 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:43:23.704 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:43:23.707 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-26T01:43:25.219 INFO:teuthology.orchestra.run.smithi045.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:43:25.271 INFO:teuthology.orchestra.run.smithi071.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:43:25.901 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:43:54.083 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-26T01:43:54.083 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:43:54.083 INFO:teuthology.orchestra.run.smithi045.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:43:54.083 INFO:teuthology.orchestra.run.smithi045.stdout: "repo_digests": [ 2024-04-26T01:43:54.084 INFO:teuthology.orchestra.run.smithi045.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:43:54.084 INFO:teuthology.orchestra.run.smithi045.stdout: ] 2024-04-26T01:43:54.084 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-26T01:43:58.376 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-26T01:43:58.376 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:43:58.377 INFO:teuthology.orchestra.run.smithi071.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:43:58.377 INFO:teuthology.orchestra.run.smithi071.stdout: "repo_digests": [ 2024-04-26T01:43:58.377 INFO:teuthology.orchestra.run.smithi071.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:43:58.377 INFO:teuthology.orchestra.run.smithi071.stdout: ] 2024-04-26T01:43:58.377 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T01:44:11.613 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T01:44:11.682 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-04-26T01:44:11.781 DEBUG:teuthology.orchestra.run.smithi045:> sudo mkdir -p /etc/ceph 2024-04-26T01:44:11.814 DEBUG:teuthology.orchestra.run.smithi071:> sudo mkdir -p /etc/ceph 2024-04-26T01:44:11.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-04-26T01:44:11.938 DEBUG:teuthology.orchestra.run.smithi045:> sudo chmod 777 /etc/ceph 2024-04-26T01:44:11.967 DEBUG:teuthology.orchestra.run.smithi071:> sudo chmod 777 /etc/ceph 2024-04-26T01:44:11.994 INFO:tasks.cephadm:Writing seed config... 2024-04-26T01:44:11.995 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T01:44:11.995 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T01:44:11.995 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T01:44:11.995 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T01:44:11.996 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T01:44:11.997 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T01:44:11.998 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T01:44:11.999 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:44:11.999 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T01:44:12.058 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 5d371fa8-036e-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T01:44:12.059 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T01:44:12.064 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T01:44:12.111 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T01:44:12.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:44:12.388 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-04-26T01:44:12.388 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', '5d371fa8-036e-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-04-26T01:44:12.389 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T01:44:12.389 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-04-26T01:44:12.449 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T01:44:12.449 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-04-26T01:44:12.450 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-04-26T01:44:12.466 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T01:44:12.466 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T01:44:12.482 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T01:44:12.482 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T01:44:12.501 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-26T01:44:12.518 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-26T01:44:12.518 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-26T01:44:12.518 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-04-26T01:44:12.583 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T01:44:12.583 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T01:44:12.583 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-04-26T01:44:12.584 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-04-26T01:44:12.600 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T01:44:12.600 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T01:44:12.616 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T01:44:12.616 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T01:44:12.638 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-04-26T01:44:12.658 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-04-26T01:44:12.659 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-04-26T01:44:12.659 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-04-26T01:44:12.659 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:12.660 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140282957200544 on /run/cephadm/5d371fa8-036e-11ef-bc93-c7b262605968.lock 2024-04-26T01:44:12.660 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140282957200544 acquired on /run/cephadm/5d371fa8-036e-11ef-bc93-c7b262605968.lock 2024-04-26T01:44:12.661 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-04-26T01:44:12.662 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-04-26T01:44:12.663 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-26T01:44:12.677 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-04-26T01:44:12.678 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-04-26T01:44:12.690 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T01:44:12.691 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-04-26T01:44:12.691 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-04-26T01:44:12.702 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T01:44:12.702 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T01:44:12.703 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T01:44:12.703 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-04-26T01:44:12.703 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T01:44:12.703 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T01:44:12.705 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-26T01:44:12.705 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-04-26T01:44:12.706 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T01:44:12.707 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T01:44:12.709 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:44:13.427 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-26T01:44:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-26T01:44:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T01:44:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-26T01:44:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-26T01:44:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-26T01:44:13.428 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T01:44:14.772 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:44:14.773 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-26T01:44:14.773 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T01:44:15.903 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-04-26T01:44:15.904 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-04-26T01:44:17.115 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDwBitmcYr5FRAAyETN6Xi02XgtL7b+ygSWgw== 2024-04-26T01:44:18.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDxBitmOStwJBAAEzGw3a2BXQofc+VmOrAoew== 2024-04-26T01:44:19.463 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQDyBitm1CbeLxAAWYLaPu3PT1XKQaTn3RyEXA== 2024-04-26T01:44:19.465 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-04-26T01:44:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:44:20.809 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-04-26T01:44:21.946 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-04-26T01:44:22.364 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T01:44:22.620 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T01:44:22.906 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-5d371fa8-036e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-5d371fa8-036e-11ef-bc93-c7b262605968.target. 2024-04-26T01:44:22.907 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-5d371fa8-036e-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-5d371fa8-036e-11ef-bc93-c7b262605968.target. 2024-04-26T01:44:23.288 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a 2024-04-26T01:44:23.289 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T01:44:23.596 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5d371fa8-036e-11ef-bc93-c7b262605968.target.wants/ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-5d371fa8-036e-11ef-bc93-c7b262605968@.service. 2024-04-26T01:44:24.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 podman[27518]: 2024-04-26 01:44:23.911126659 +0000 UTC m=+0.038309627 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:44:24.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 podman[27518]: 2024-04-26 01:44:24.278643483 +0000 UTC m=+0.405826481 container init e55db2404f3b64feb2b60f921a82aded636f5ca4849484bd4b89b61783ccec19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T01:44:24.401 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:44:24.401 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T01:44:24.419 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:44:24.419 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T01:44:24.419 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T01:44:24.419 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T01:44:24.420 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-04-26T01:44:24.420 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-04-26T01:44:24.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 podman[27518]: 2024-04-26 01:44:24.290088027 +0000 UTC m=+0.417271018 container start e55db2404f3b64feb2b60f921a82aded636f5ca4849484bd4b89b61783ccec19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:44:24.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 bash[27518]: e55db2404f3b64feb2b60f921a82aded636f5ca4849484bd4b89b61783ccec19 2024-04-26T01:44:24.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 ceph-mon[27537]: mkfs 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:24.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 ceph-mon[27537]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:44:24.574 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:24 smithi037 systemd[1]: Started Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: monmap epoch 1 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: last_changed 2024-04-26T01:44:19.989980+0000 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: min_mon_release 19 (squid) 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: election_strategy: 1 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:44:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: fsmap 2024-04-26T01:44:25.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T01:44:25.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: mgrmap e1: no daemons active 2024-04-26T01:44:25.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:25 smithi037 ceph-mon[27537]: from='client.? 172.21.15.37:0/107241521' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T01:44:25.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T01:44:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.745462s) 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T01:44:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.018+0000 7f17d7800700 1 Processor -- start 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.018+0000 7f17d7800700 1 -- start start 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.018+0000 7f17d7800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d010f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.018+0000 7f17d7800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17d010fa30 con 0x7f17d010f040 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.020+0000 7f17d4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d010f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.020+0000 7f17d4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d010f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58456/0 (socket says 172.21.15.37:58456) 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.020+0000 7f17d4e00700 1 -- 172.21.15.37:0/1888928145 learned_addr learned my addr 172.21.15.37:0/1888928145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.020+0000 7f17d4e00700 1 -- 172.21.15.37:0/1888928145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d010fdc0 con 0x7f17d010f040 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.020+0000 7f17d4e00700 1 --2- 172.21.15.37:0/1888928145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d010f460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e80 tx=0x7f17c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303e12f635a611e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.021+0000 7f17cf400700 1 -- 172.21.15.37:0/1888928145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4017070 con 0x7f17d010f040 2024-04-26T01:44:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.021+0000 7f17cf400700 1 -- 172.21.15.37:0/1888928145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4003a20 con 0x7f17d010f040 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.021+0000 7f17cf400700 1 -- 172.21.15.37:0/1888928145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4003d20 con 0x7f17d010f040 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.021+0000 7f17d7800700 1 -- 172.21.15.37:0/1888928145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 msgr2=0x7f17d010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.021+0000 7f17d7800700 1 --2- 172.21.15.37:0/1888928145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d010f460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17c4009e80 tx=0x7f17c400b330 comp rx=0 tx=0).stop 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.022+0000 7f17d7800700 1 -- 172.21.15.37:0/1888928145 shutdown_connections 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.022+0000 7f17d7800700 1 --2- 172.21.15.37:0/1888928145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d010f460 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.022+0000 7f17d7800700 1 -- 172.21.15.37:0/1888928145 >> 172.21.15.37:0/1888928145 conn(0x7f17d007ddf0 msgr2=0x7f17d0080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.022+0000 7f17d7800700 1 -- 172.21.15.37:0/1888928145 shutdown_connections 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.022+0000 7f17d7800700 1 -- 172.21.15.37:0/1888928145 wait complete. 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.023+0000 7f17d7800700 1 Processor -- start 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.023+0000 7f17d7800700 1 -- start start 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.023+0000 7f17d7800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d01a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.023+0000 7f17d7800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17d01102f0 con 0x7f17d010f040 2024-04-26T01:44:25.826 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.024+0000 7f17d4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d01a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.024+0000 7f17d4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d01a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58468/0 (socket says 172.21.15.37:58468) 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.024+0000 7f17d4e00700 1 -- 172.21.15.37:0/107241521 learned_addr learned my addr 172.21.15.37:0/107241521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.024+0000 7f17d4e00700 1 -- 172.21.15.37:0/107241521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17d01a2730 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.024+0000 7f17d4e00700 1 --2- 172.21.15.37:0/107241521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d01a21f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f17d00822c0 tx=0x7f17c4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4022020 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17c4018040 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f17c4017070 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17d01a29c0 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17d01a56b0 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f17c4025020 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.025+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f17c400f050 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.026+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17d00640a0 con 0x7f17d010f040 2024-04-26T01:44:25.827 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.029+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f17c402c030 con 0x7f17d010f040 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.110+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f17d002e060 con 0x7f17d010f040 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.111+0000 7f17cd600700 1 -- 172.21.15.37:0/107241521 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f17c401e480 con 0x7f17d010f040 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 msgr2=0x7f17d01a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 --2- 172.21.15.37:0/107241521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d01a21f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f17d00822c0 tx=0x7f17c4004ad0 comp rx=0 tx=0).stop 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 shutdown_connections 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 --2- 172.21.15.37:0/107241521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17d010f040 0x7f17d01a21f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 >> 172.21.15.37:0/107241521 conn(0x7f17d007ddf0 msgr2=0x7f17d0198ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 shutdown_connections 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:25.114+0000 7f17d7800700 1 -- 172.21.15.37:0/107241521 wait complete. 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-04-26T01:44:25.828 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T01:44:26.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:26 smithi037 ceph-mon[27537]: from='client.? 172.21.15.37:0/4105209194' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T01:44:26.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:26 smithi037 ceph-mon[27537]: from='client.? 172.21.15.37:0/4105209194' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T01:44:27.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:27.185 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T01:44:27.186 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:27.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T01:44:27.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T01:44:27.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:27.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T01:44:27.192 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.381+0000 7f2245600700 1 Processor -- start 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.381+0000 7f2245600700 1 -- start start 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.381+0000 7f2245600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f224010f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.382+0000 7f2245600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f224010fa30 con 0x7f224010f040 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.382+0000 7f223ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f224010f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.382+0000 7f223ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f224010f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58470/0 (socket says 172.21.15.37:58470) 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.382+0000 7f223ea00700 1 -- 172.21.15.37:0/3937696298 learned_addr learned my addr 172.21.15.37:0/3937696298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.383+0000 7f223ea00700 1 -- 172.21.15.37:0/3937696298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224010fdc0 con 0x7f224010f040 2024-04-26T01:44:27.193 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.383+0000 7f223ea00700 1 --2- 172.21.15.37:0/3937696298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f224010f460 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f22300098a0 tx=0x7f2230009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8ece8452f947aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.383+0000 7f223d600700 1 -- 172.21.15.37:0/3937696298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2230004160 con 0x7f224010f040 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.383+0000 7f223d600700 1 -- 172.21.15.37:0/3937696298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2230004300 con 0x7f224010f040 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.383+0000 7f223d600700 1 -- 172.21.15.37:0/3937696298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22300036a0 con 0x7f224010f040 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.384+0000 7f2245600700 1 -- 172.21.15.37:0/3937696298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 msgr2=0x7f224010f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.384+0000 7f2245600700 1 --2- 172.21.15.37:0/3937696298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f224010f460 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f22300098a0 tx=0x7f2230009bb0 comp rx=0 tx=0).stop 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.384+0000 7f2245600700 1 -- 172.21.15.37:0/3937696298 shutdown_connections 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.384+0000 7f2245600700 1 --2- 172.21.15.37:0/3937696298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f224010f460 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.384+0000 7f2245600700 1 -- 172.21.15.37:0/3937696298 >> 172.21.15.37:0/3937696298 conn(0x7f224007ddf0 msgr2=0x7f2240080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.384+0000 7f2245600700 1 -- 172.21.15.37:0/3937696298 shutdown_connections 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.385+0000 7f2245600700 1 -- 172.21.15.37:0/3937696298 wait complete. 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.386+0000 7f2245600700 1 Processor -- start 2024-04-26T01:44:27.194 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.386+0000 7f2245600700 1 -- start start 2024-04-26T01:44:27.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.386+0000 7f2245600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f22401a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:27.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.386+0000 7f2245600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f22401102f0 con 0x7f224010f040 2024-04-26T01:44:27.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.387+0000 7f223ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f22401a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:27.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.387+0000 7f223ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f22401a21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58480/0 (socket says 172.21.15.37:58480) 2024-04-26T01:44:27.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.387+0000 7f223ea00700 1 -- 172.21.15.37:0/4105209194 learned_addr learned my addr 172.21.15.37:0/4105209194 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:27.197 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.387+0000 7f223ea00700 1 -- 172.21.15.37:0/4105209194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22401a2730 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.387+0000 7f223ea00700 1 --2- 172.21.15.37:0/4105209194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f22401a21f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2230000c00 tx=0x7f2230004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2230003db0 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2230011630 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22401a29c0 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2230011950 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22401a56b0 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2230011af0 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.388+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f223001c450 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.389+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22400640a0 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.392+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f2230032020 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.470+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f224002e080 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.478+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f2230004260 con 0x7f224010f040 2024-04-26T01:44:27.198 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.479+0000 7f2237400700 1 -- 172.21.15.37:0/4105209194 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f223001c650 con 0x7f224010f040 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.480+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 msgr2=0x7f22401a21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.480+0000 7f2245600700 1 --2- 172.21.15.37:0/4105209194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f22401a21f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f2230000c00 tx=0x7f2230004750 comp rx=0 tx=0).stop 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.480+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 shutdown_connections 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.480+0000 7f2245600700 1 --2- 172.21.15.37:0/4105209194 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f224010f040 0x7f22401a21f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.480+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 >> 172.21.15.37:0/4105209194 conn(0x7f224007ddf0 msgr2=0x7f2240198ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.481+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 shutdown_connections 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:26.481+0000 7f2245600700 1 -- 172.21.15.37:0/4105209194 wait complete. 2024-04-26T01:44:27.199 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-04-26T01:44:28.206 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:27 smithi037 ceph-mon[27537]: from='client.? 172.21.15.37:0/693913728' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:44:28.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.840+0000 7efeb4200700 1 Processor -- start 2024-04-26T01:44:28.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.840+0000 7efeb4200700 1 -- start start 2024-04-26T01:44:28.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.840+0000 7efeb4200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac10f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:28.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.841+0000 7efeb4200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efeac10fa30 con 0x7efeac10f040 2024-04-26T01:44:28.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.841+0000 7efeb1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:28.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.841+0000 7efeb1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac10f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58486/0 (socket says 172.21.15.37:58486) 2024-04-26T01:44:28.665 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.841+0000 7efeb1800700 1 -- 172.21.15.37:0/2459140246 learned_addr learned my addr 172.21.15.37:0/2459140246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.842+0000 7efeb1800700 1 -- 172.21.15.37:0/2459140246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac10fdc0 con 0x7efeac10f040 2024-04-26T01:44:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.842+0000 7efeb1800700 1 --2- 172.21.15.37:0/2459140246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac10f460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efea0009e80 tx=0x7efea000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9642e2bea95fade3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.842+0000 7efeabe00700 1 -- 172.21.15.37:0/2459140246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea0017070 con 0x7efeac10f040 2024-04-26T01:44:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.842+0000 7efeabe00700 1 -- 172.21.15.37:0/2459140246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efea000b8c0 con 0x7efeac10f040 2024-04-26T01:44:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.842+0000 7efeabe00700 1 -- 172.21.15.37:0/2459140246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea0004490 con 0x7efeac10f040 2024-04-26T01:44:28.670 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 -- 172.21.15.37:0/2459140246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 msgr2=0x7efeac10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 --2- 172.21.15.37:0/2459140246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac10f460 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efea0009e80 tx=0x7efea000b330 comp rx=0 tx=0).stop 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 -- 172.21.15.37:0/2459140246 shutdown_connections 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 --2- 172.21.15.37:0/2459140246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac10f460 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 -- 172.21.15.37:0/2459140246 >> 172.21.15.37:0/2459140246 conn(0x7efeac07ddf0 msgr2=0x7efeac080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 -- 172.21.15.37:0/2459140246 shutdown_connections 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.843+0000 7efeb4200700 1 -- 172.21.15.37:0/2459140246 wait complete. 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.844+0000 7efeb4200700 1 Processor -- start 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.844+0000 7efeb4200700 1 -- start start 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.845+0000 7efeb4200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac1a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.845+0000 7efeb4200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efeac112ff0 con 0x7efeac10f040 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.845+0000 7efeb1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac1a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.845+0000 7efeb1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac1a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58490/0 (socket says 172.21.15.37:58490) 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.845+0000 7efeb1800700 1 -- 172.21.15.37:0/693913728 learned_addr learned my addr 172.21.15.37:0/693913728 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:28.671 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.846+0000 7efeb1800700 1 -- 172.21.15.37:0/693913728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac1a2f40 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.846+0000 7efeb1800700 1 --2- 172.21.15.37:0/693913728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac1a2a00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efea000b7b0 tx=0x7efea000be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.846+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea0020020 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.847+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efea0003d10 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.847+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac1a31d0 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.847+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efea0017040 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.847+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1a5ec0 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.847+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efea0023020 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.847+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efea000f050 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.848+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeac113170 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.851+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7efea002a020 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.929+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7efeac0640a0 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.929+0000 7efeaa000700 1 -- 172.21.15.37:0/693913728 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7efea0003ec0 con 0x7efeac10f040 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.931+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 msgr2=0x7efeac1a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:28.672 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.931+0000 7efeb4200700 1 --2- 172.21.15.37:0/693913728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac1a2a00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efea000b7b0 tx=0x7efea000be90 comp rx=0 tx=0).stop 2024-04-26T01:44:28.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.931+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 shutdown_connections 2024-04-26T01:44:28.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.932+0000 7efeb4200700 1 --2- 172.21.15.37:0/693913728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efeac10f040 0x7efeac1a2a00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:28.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.932+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 >> 172.21.15.37:0/693913728 conn(0x7efeac07ddf0 msgr2=0x7efeac07f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:28.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.932+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 shutdown_connections 2024-04-26T01:44:28.673 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:27.932+0000 7efeb4200700 1 -- 172.21.15.37:0/693913728 wait complete. 2024-04-26T01:44:28.673 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-04-26T01:44:28.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:28 smithi037 systemd[1]: Stopping Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:44:29.683 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:29 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a[27533]: 2024-04-26T01:44:29.208+0000 7f84aca00700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T01:44:29.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:29 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a[27533]: 2024-04-26T01:44:29.208+0000 7f84aca00700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T01:44:29.684 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:29 smithi037 podman[27780]: 2024-04-26 01:44:29.374134093 +0000 UTC m=+0.310478732 container died e55db2404f3b64feb2b60f921a82aded636f5ca4849484bd4b89b61783ccec19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T01:44:34.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:33 smithi037 podman[27780]: 2024-04-26 01:44:33.864828381 +0000 UTC m=+4.801173050 container cleanup e55db2404f3b64feb2b60f921a82aded636f5ca4849484bd4b89b61783ccec19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:44:34.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:33 smithi037 bash[27780]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a 2024-04-26T01:44:38.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 podman[27794]: 2024-04-26 01:44:38.265176215 +0000 UTC m=+8.870561823 container remove e55db2404f3b64feb2b60f921a82aded636f5ca4849484bd4b89b61783ccec19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-26T01:44:38.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T01:44:38.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 systemd[1]: Stopped Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:44:38.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service: Consumed 1.961s CPU time. 2024-04-26T01:44:38.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 systemd[1]: Starting Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:44:39.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 podman[27887]: 2024-04-26 01:44:38.872092082 +0000 UTC m=+0.042251579 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:44:39.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:38 smithi037 podman[27887]: 2024-04-26 01:44:38.966059843 +0000 UTC m=+0.136219332 container create d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:44:39.217 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 podman[27887]: 2024-04-26 01:44:39.219753192 +0000 UTC m=+0.389912730 container init d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T01:44:39.316 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 podman[27887]: 2024-04-26 01:44:39.230819494 +0000 UTC m=+0.400978961 container start d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: pidfile_write: ignore empty --pid-file 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: load: jerasure load: lrc 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: RocksDB version: 7.9.2 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Git sha 0 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-26T01:44:39.484 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: DB SUMMARY 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: DB Session ID: Q7P7BQO0V6WAQ64BQLN2 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: CURRENT file: CURRENT 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104882 ; 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.error_if_exists: 0 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.create_if_missing: 0 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.paranoid_checks: 1 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T01:44:39.485 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.env: 0x559b65fcfcc0 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.info_log: 0x559ba369bc40 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.statistics: (nil) 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.use_fsync: 0 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_log_file_size: 0 2024-04-26T01:44:39.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.allow_fallocate: 1 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T01:44:39.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.use_direct_reads: 0 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.db_log_dir: 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.wal_dir: 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T01:44:39.488 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.write_buffer_manager: 0x559ba362f0e0 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T01:44:39.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.unordered_write: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.row_cache: None 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.wal_filter: None 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.two_write_queues: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.wal_compression: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.atomic_flush: 0 2024-04-26T01:44:39.490 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.log_readahead_size: 0 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_background_jobs: 2 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_background_compactions: -1 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_subcompactions: 1 2024-04-26T01:44:39.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_open_files: -1 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_background_flushes: -1 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Compression algorithms supported: 2024-04-26T01:44:39.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kZSTD supported: 0 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kXpressCompression supported: 0 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kZlibCompression supported: 1 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kSnappyCompression supported: 1 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kLZ4Compression supported: 1 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: kBZip2Compression supported: 0 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T01:44:39.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.merge_operator: 2024-04-26T01:44:39.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_filter: None 2024-04-26T01:44:39.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559ba369be00) 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T01:44:39.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x559ba36611f0 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-04-26T01:44:39.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-04-26T01:44:39.497 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression: NoCompression 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.num_levels: 7 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T01:44:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T01:44:39.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T01:44:39.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:44:39.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T01:44:39.500 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T01:44:39.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T01:44:39.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T01:44:39.503 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.inplace_update_support: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.bloom_locality: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.max_successive_merges: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.ttl: 2592000 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T01:44:39.504 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.enable_blob_files: false 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.min_blob_size: 0 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: c3dafeda-ca23-4e41-ad7e-ce995583ee5c 2024-04-26T01:44:39.505 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095879293207, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095879295064, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100030, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 97912, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14131, "raw_average_key_size": 49, "raw_value_size": 90256, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714095879, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "c3dafeda-ca23-4e41-ad7e-ce995583ee5c", "db_session_id": "Q7P7BQO0V6WAQ64BQLN2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095879295234, "job": 1, "event": "recovery_finished"} 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559ba3776000 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: DB pointer 0x559ba3760000 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:44:39.506 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 99.55 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 99.55 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 75.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:44:39.507 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 75.9 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T01:44:39.508 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 13.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 13.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x559ba36611f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???) e1 preinit fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).mds e1 new map 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).mds e1 print_map 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:44:39.509 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 bash[27887]: d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 systemd[1]: Started Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: monmap epoch 1 2024-04-26T01:44:39.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: last_changed 2024-04-26T01:44:19.989980+0000 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: min_mon_release 19 (squid) 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: election_strategy: 1 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: fsmap 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T01:44:39.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:39 smithi037 ceph-mon[27906]: mgrmap e1: no daemons active 2024-04-26T01:44:40.780 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.048+0000 7f9249400700 1 Processor -- start 2024-04-26T01:44:40.780 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.048+0000 7f9249400700 1 -- start start 2024-04-26T01:44:40.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.048+0000 7f9249400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f924410f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:40.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.048+0000 7f9249400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f924410fa30 con 0x7f924410f040 2024-04-26T01:44:40.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.049+0000 7f9242800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f924410f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:40.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.049+0000 7f9242800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f924410f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59598/0 (socket says 172.21.15.37:59598) 2024-04-26T01:44:40.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.049+0000 7f9242800700 1 -- 172.21.15.37:0/2370853315 learned_addr learned my addr 172.21.15.37:0/2370853315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.050+0000 7f9242800700 1 -- 172.21.15.37:0/2370853315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924410fdc0 con 0x7f924410f040 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.050+0000 7f9242800700 1 --2- 172.21.15.37:0/2370853315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f924410f460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9234009e80 tx=0x7f923400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fafd9b6a53df60e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.050+0000 7f9241e00700 1 -- 172.21.15.37:0/2370853315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9234017070 con 0x7f924410f040 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.050+0000 7f9241e00700 1 -- 172.21.15.37:0/2370853315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f923400b8c0 con 0x7f924410f040 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.050+0000 7f9241e00700 1 -- 172.21.15.37:0/2370853315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9234004490 con 0x7f924410f040 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 -- 172.21.15.37:0/2370853315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 msgr2=0x7f924410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 --2- 172.21.15.37:0/2370853315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f924410f460 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9234009e80 tx=0x7f923400b330 comp rx=0 tx=0).stop 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 -- 172.21.15.37:0/2370853315 shutdown_connections 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 --2- 172.21.15.37:0/2370853315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f924410f460 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 -- 172.21.15.37:0/2370853315 >> 172.21.15.37:0/2370853315 conn(0x7f924407ddf0 msgr2=0x7f9244080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 -- 172.21.15.37:0/2370853315 shutdown_connections 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.051+0000 7f9249400700 1 -- 172.21.15.37:0/2370853315 wait complete. 2024-04-26T01:44:40.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.053+0000 7f9249400700 1 Processor -- start 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.053+0000 7f9249400700 1 -- start start 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.053+0000 7f9249400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f92441a2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.053+0000 7f9249400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9244112ff0 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.053+0000 7f9242800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f92441a2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.054+0000 7f9242800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f92441a2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59606/0 (socket says 172.21.15.37:59606) 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.054+0000 7f9242800700 1 -- 172.21.15.37:0/2961944951 learned_addr learned my addr 172.21.15.37:0/2961944951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.054+0000 7f9242800700 1 -- 172.21.15.37:0/2961944951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92441a2f40 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.054+0000 7f9242800700 1 --2- 172.21.15.37:0/2961944951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f92441a2a00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f923400b7b0 tx=0x7f923400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9234020020 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92441a31d0 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9234012430 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92441a5ec0 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9234017040 con 0x7f924410f040 2024-04-26T01:44:40.788 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9234023020 con 0x7f924410f040 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.055+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f923400f050 con 0x7f924410f040 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.056+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9244113170 con 0x7f924410f040 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.060+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f923402a020 con 0x7f924410f040 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.138+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f92441a6090 con 0x7f924410f040 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.142+0000 7f923be00700 1 -- 172.21.15.37:0/2961944951 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f923400b860 con 0x7f924410f040 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.143+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 msgr2=0x7f92441a2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.143+0000 7f9249400700 1 --2- 172.21.15.37:0/2961944951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f92441a2a00 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f923400b7b0 tx=0x7f923400be20 comp rx=0 tx=0).stop 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.144+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 shutdown_connections 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.144+0000 7f9249400700 1 --2- 172.21.15.37:0/2961944951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f924410f040 0x7f92441a2a00 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.144+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 >> 172.21.15.37:0/2961944951 conn(0x7f924407ddf0 msgr2=0x7f924407f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.144+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 shutdown_connections 2024-04-26T01:44:40.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:40.144+0000 7f9249400700 1 -- 172.21.15.37:0/2961944951 wait complete. 2024-04-26T01:44:40.790 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T01:44:40.813 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:44:40.813 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-04-26T01:44:40.815 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T01:44:40.816 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T01:44:41.312 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x 2024-04-26T01:44:41.312 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T01:44:41.456 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:41 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2961944951' entity='client.admin' 2024-04-26T01:44:41.594 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-5d371fa8-036e-11ef-bc93-c7b262605968.target.wants/ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-5d371fa8-036e-11ef-bc93-c7b262605968@.service. 2024-04-26T01:44:41.957 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:41 smithi037 systemd[1]: Starting Ceph mgr.x for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:44:42.279 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 podman[28123]: 2024-04-26 01:44:41.929291879 +0000 UTC m=+0.034030857 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-26T01:44:42.279 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 podman[28123]: 2024-04-26 01:44:42.052253151 +0000 UTC m=+0.156992154 container create 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T01:44:42.279 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 podman[28123]: 2024-04-26 01:44:42.280461752 +0000 UTC m=+0.385200762 container init 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T01:44:42.403 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:44:42.403 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T01:44:42.419 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:44:42.419 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T01:44:42.420 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T01:44:42.420 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T01:44:42.445 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:44:42.445 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T01:44:42.464 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:44:42.464 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T01:44:42.464 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T01:44:42.465 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T01:44:42.465 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-04-26T01:44:42.465 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-04-26T01:44:42.532 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 podman[28123]: 2024-04-26 01:44:42.291969093 +0000 UTC m=+0.396708065 container start 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux ) 2024-04-26T01:44:42.532 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 bash[28123]: 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 2024-04-26T01:44:42.532 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 systemd[1]: Started Ceph mgr.x for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:44:42.796 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:42.534+0000 7fe9e59da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:44:43.154 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:42.798+0000 7fe9e59da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:44:43.154 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:42.937+0000 7fe9e59da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:44:43.404 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:43.386+0000 7fe9e59da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:44:43.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:43 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3638479420' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:44:43.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:43.510+0000 7fe9e59da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:44:44.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:44.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:44:44.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:44:44.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:44:44.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:44:44.512 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:44:44.513 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:44:44.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:44:44.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:44.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:44:44.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:44:44.514 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:44:44.522 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:44:44.523 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:44:24.369478+0000", 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:44:44.524 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.483+0000 7fd012800700 1 Processor -- start 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.484+0000 7fd012800700 1 -- start start 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.484+0000 7fd012800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c07b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.484+0000 7fd012800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c07b7e0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.484+0000 7fd00b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c07b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.485+0000 7fd00b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c07b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59642/0 (socket says 172.21.15.37:59642) 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.485+0000 7fd00b400700 1 -- 172.21.15.37:0/633367412 learned_addr learned my addr 172.21.15.37:0/633367412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.485+0000 7fd00b400700 1 -- 172.21.15.37:0/633367412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c07b960 con 0x7fd00c07cdb0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.486+0000 7fd00b400700 1 --2- 172.21.15.37:0/633367412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c07b210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0098a0 tx=0x7fcffc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f65d4e48f62d62f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.486+0000 7fd00a000700 1 -- 172.21.15.37:0/633367412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcffc004160 con 0x7fd00c07cdb0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.486+0000 7fd00a000700 1 -- 172.21.15.37:0/633367412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcffc004300 con 0x7fd00c07cdb0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.486+0000 7fd00a000700 1 -- 172.21.15.37:0/633367412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcffc011af0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.487+0000 7fd012800700 1 -- 172.21.15.37:0/633367412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 msgr2=0x7fd00c07b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.487+0000 7fd012800700 1 --2- 172.21.15.37:0/633367412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c07b210 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fcffc0098a0 tx=0x7fcffc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.525 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.487+0000 7fd012800700 1 -- 172.21.15.37:0/633367412 shutdown_connections 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.487+0000 7fd012800700 1 --2- 172.21.15.37:0/633367412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c07b210 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.487+0000 7fd012800700 1 -- 172.21.15.37:0/633367412 >> 172.21.15.37:0/633367412 conn(0x7fd00c103bf0 msgr2=0x7fd00c106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.488+0000 7fd012800700 1 -- 172.21.15.37:0/633367412 shutdown_connections 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.488+0000 7fd012800700 1 -- 172.21.15.37:0/633367412 wait complete. 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.489+0000 7fd012800700 1 Processor -- start 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.489+0000 7fd012800700 1 -- start start 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.490+0000 7fd012800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c1a2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.490+0000 7fd012800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd00c10a720 con 0x7fd00c07cdb0 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.490+0000 7fd00b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c1a2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.490+0000 7fd00b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c1a2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59648/0 (socket says 172.21.15.37:59648) 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.490+0000 7fd00b400700 1 -- 172.21.15.37:0/3638479420 learned_addr learned my addr 172.21.15.37:0/3638479420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.490+0000 7fd00b400700 1 -- 172.21.15.37:0/3638479420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd00c1a2fe0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.526 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.491+0000 7fd00b400700 1 --2- 172.21.15.37:0/3638479420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c1a2aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcffc003f40 tx=0x7fcffc011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.491+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcffc0038b0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.491+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcffc024430 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.491+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcffc01a430 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.491+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd00c1a3270 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.491+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd00c1a5f60 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.492+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcffc01aa70 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.492+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcffc0193c0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.493+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd00c0640a0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.495+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fcffc043e50 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.573+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd00c02e1f0 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.574+0000 7fd003e00700 1 -- 172.21.15.37:0/3638479420 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fcffc032030 con 0x7fd00c07cdb0 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 msgr2=0x7fd00c1a2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:44.527 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 --2- 172.21.15.37:0/3638479420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c1a2aa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fcffc003f40 tx=0x7fcffc011890 comp rx=0 tx=0).stop 2024-04-26T01:44:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 shutdown_connections 2024-04-26T01:44:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 --2- 172.21.15.37:0/3638479420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd00c07cdb0 0x7fd00c1a2aa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 >> 172.21.15.37:0/3638479420 conn(0x7fd00c103bf0 msgr2=0x7fd00c105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 shutdown_connections 2024-04-26T01:44:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:43.576+0000 7fd012800700 1 -- 172.21.15.37:0/3638479420 wait complete. 2024-04-26T01:44:44.528 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-04-26T01:44:45.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:45 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:45.159+0000 7fe9e59da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:44:45.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:45 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:45.392+0000 7fe9e59da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:44:45.956 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:45 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:45.608+0000 7fe9e59da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:44:46.377 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:46 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:45.999+0000 7fe9e59da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:44:46.377 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:46 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:46.124+0000 7fe9e59da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:44:46.642 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:46 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:46.379+0000 7fe9e59da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:44:46.642 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:46 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:46.644+0000 7fe9e59da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:44:47.591 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:47 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3481255118' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:44:47.591 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:47 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:47.494+0000 7fe9e59da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:44:47.904 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:47.906 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:44:47.906 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:44:47.906 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:44:47.907 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:44:47.908 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:44:47.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:44:47.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:44:24.369478+0000", 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:47.918 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.163+0000 7f5491a00700 1 Processor -- start 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.163+0000 7f5491a00700 1 -- start start 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.164+0000 7f5491a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c109280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.164+0000 7f5491a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f548c063d40 con 0x7f548c106e50 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.164+0000 7f548aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c109280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.164+0000 7f548aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c109280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59656/0 (socket says 172.21.15.37:59656) 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.164+0000 7f548aa00700 1 -- 172.21.15.37:0/1681874092 learned_addr learned my addr 172.21.15.37:0/1681874092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.165+0000 7f548aa00700 1 -- 172.21.15.37:0/1681874092 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c1097c0 con 0x7f548c106e50 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.165+0000 7f548aa00700 1 --2- 172.21.15.37:0/1681874092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c109280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5484009e80 tx=0x7f548400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9c183471843666 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.165+0000 7f5489600700 1 -- 172.21.15.37:0/1681874092 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484017070 con 0x7f548c106e50 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.165+0000 7f5489600700 1 -- 172.21.15.37:0/1681874092 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f548400b8c0 con 0x7f548c106e50 2024-04-26T01:44:47.919 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.165+0000 7f5489600700 1 -- 172.21.15.37:0/1681874092 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484004490 con 0x7f548c106e50 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 -- 172.21.15.37:0/1681874092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 msgr2=0x7f548c109280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 --2- 172.21.15.37:0/1681874092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c109280 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5484009e80 tx=0x7f548400b330 comp rx=0 tx=0).stop 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 -- 172.21.15.37:0/1681874092 shutdown_connections 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 --2- 172.21.15.37:0/1681874092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c109280 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 -- 172.21.15.37:0/1681874092 >> 172.21.15.37:0/1681874092 conn(0x7f548c102ab0 msgr2=0x7f548c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 -- 172.21.15.37:0/1681874092 shutdown_connections 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.166+0000 7f5491a00700 1 -- 172.21.15.37:0/1681874092 wait complete. 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.167+0000 7f5491a00700 1 Processor -- start 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.167+0000 7f5491a00700 1 -- start start 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.168+0000 7f5491a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c1a2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.168+0000 7f5491a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f548c10a780 con 0x7f548c106e50 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.168+0000 7f548aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c1a2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.168+0000 7f548aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c1a2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59664/0 (socket says 172.21.15.37:59664) 2024-04-26T01:44:47.920 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.168+0000 7f548aa00700 1 -- 172.21.15.37:0/3481255118 learned_addr learned my addr 172.21.15.37:0/3481255118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.168+0000 7f548aa00700 1 -- 172.21.15.37:0/3481255118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f548c1a2f80 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f548aa00700 1 --2- 172.21.15.37:0/3481255118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c1a2a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f548400b7b0 tx=0x7f548400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484020020 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5484012430 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f548c1a3210 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5484017040 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f548c1a5f00 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.169+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5484023020 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.170+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f548400f050 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.170+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f548c0640a0 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.173+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f548402a020 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.253+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f548c02e080 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.253+0000 7f5483400700 1 -- 172.21.15.37:0/3481255118 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5484012690 con 0x7f548c106e50 2024-04-26T01:44:47.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 msgr2=0x7f548c1a2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 --2- 172.21.15.37:0/3481255118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c1a2a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f548400b7b0 tx=0x7f548400be20 comp rx=0 tx=0).stop 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 shutdown_connections 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 --2- 172.21.15.37:0/3481255118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f548c106e50 0x7f548c1a2a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 >> 172.21.15.37:0/3481255118 conn(0x7f548c102ab0 msgr2=0x7f548c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 shutdown_connections 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:47.255+0000 7f5491a00700 1 -- 172.21.15.37:0/3481255118 wait complete. 2024-04-26T01:44:47.922 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-04-26T01:44:48.192 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:48.065+0000 7fe9e59da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:44:48.456 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:48.195+0000 7fe9e59da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:44:48.956 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:48.466+0000 7fe9e59da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:44:49.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:49 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:49.129+0000 7fe9e59da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:44:49.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:49 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:49.256+0000 7fe9e59da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:44:49.898 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:49 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:49.588+0000 7fe9e59da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:44:49.899 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:49 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:49.771+0000 7fe9e59da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:44:49.899 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:49 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:49.901+0000 7fe9e59da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:44:50.191 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:50.056+0000 7fe9e59da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:44:50.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:50.442+0000 7fe9e59da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:44:50.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:50 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2944483370' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:44:51.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:51.233 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:51.234 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:44:51.235 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:44:51.243 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:51.244 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:44:24.369478+0000", 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.490+0000 7f1a62400700 1 Processor -- start 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.491+0000 7f1a62400700 1 -- start start 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.491+0000 7f1a62400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c07b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.491+0000 7f1a62400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c07b7a0 con 0x7f1a5c07cd70 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.491+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c07b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.492+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c07b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44040/0 (socket says 172.21.15.37:44040) 2024-04-26T01:44:51.245 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.492+0000 7f1a5b400700 1 -- 172.21.15.37:0/703096218 learned_addr learned my addr 172.21.15.37:0/703096218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.492+0000 7f1a5b400700 1 -- 172.21.15.37:0/703096218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c07b920 con 0x7f1a5c07cd70 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.492+0000 7f1a5b400700 1 --2- 172.21.15.37:0/703096218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c07b1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55b29ef22cf9681a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a5a000700 1 -- 172.21.15.37:0/703096218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a4c004160 con 0x7f1a5c07cd70 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a5a000700 1 -- 172.21.15.37:0/703096218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a4c004300 con 0x7f1a5c07cd70 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a5a000700 1 -- 172.21.15.37:0/703096218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a4c011af0 con 0x7f1a5c07cd70 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a62400700 1 -- 172.21.15.37:0/703096218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 msgr2=0x7f1a5c07b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a62400700 1 --2- 172.21.15.37:0/703096218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c07b1d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098a0 tx=0x7f1a4c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a62400700 1 -- 172.21.15.37:0/703096218 shutdown_connections 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a62400700 1 --2- 172.21.15.37:0/703096218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c07b1d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a62400700 1 -- 172.21.15.37:0/703096218 >> 172.21.15.37:0/703096218 conn(0x7f1a5c103be0 msgr2=0x7f1a5c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.493+0000 7f1a62400700 1 -- 172.21.15.37:0/703096218 shutdown_connections 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.494+0000 7f1a62400700 1 -- 172.21.15.37:0/703096218 wait complete. 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a62400700 1 Processor -- start 2024-04-26T01:44:51.246 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a62400700 1 -- start start 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a62400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a62400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a5c10eb10 con 0x7f1a5c07cd70 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a5b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44050/0 (socket says 172.21.15.37:44050) 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.495+0000 7f1a5b400700 1 -- 172.21.15.37:0/2944483370 learned_addr learned my addr 172.21.15.37:0/2944483370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a5b400700 1 -- 172.21.15.37:0/2944483370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a5c19ec80 con 0x7f1a5c07cd70 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a5b400700 1 --2- 172.21.15.37:0/2944483370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c19e740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fb0 tx=0x7f1a4c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a4c0036a0 con 0x7f1a5c07cd70 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1a4c01a430 con 0x7f1a5c07cd70 2024-04-26T01:44:51.247 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1a4c019550 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a5c19ef10 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.496+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a5c1a1c00 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.497+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1a4c019be0 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.497+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a4c024730 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.497+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a5c0640a0 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.500+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f1a4c043e50 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.582+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f1a5c1a20a0 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.583+0000 7f1a53e00700 1 -- 172.21.15.37:0/2944483370 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1240 (secure 0 0 0) 0x7f1a4c032020 con 0x7f1a5c07cd70 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 msgr2=0x7f1a5c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 --2- 172.21.15.37:0/2944483370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c19e740 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c009fb0 tx=0x7f1a4c011940 comp rx=0 tx=0).stop 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 shutdown_connections 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 --2- 172.21.15.37:0/2944483370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a5c07cd70 0x7f1a5c19e740 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 >> 172.21.15.37:0/2944483370 conn(0x7f1a5c103be0 msgr2=0x7f1a5c1047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:51.248 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 shutdown_connections 2024-04-26T01:44:51.249 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:50.584+0000 7f1a62400700 1 -- 172.21.15.37:0/2944483370 wait complete. 2024-04-26T01:44:51.249 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (3/15)... 2024-04-26T01:44:51.526 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:51.239+0000 7fe9e59da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:44:51.956 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:51.529+0000 7fe9e59da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: Activating manager daemon x 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: mgrmap e2: x(active, starting, since 0.01204s) 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:44:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: Manager daemon x is now available 2024-04-26T01:44:51.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:44:51.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' 2024-04-26T01:44:51.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:44:51.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' 2024-04-26T01:44:51.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:51 smithi037 ceph-mon[27906]: from='mgr.14100 172.21.15.37:0/970363216' entity='mgr.x' 2024-04-26T01:44:53.814 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:53 smithi037 ceph-mon[27906]: mgrmap e3: x(active, since 1.02407s) 2024-04-26T01:44:54.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:54 smithi037 ceph-mon[27906]: mgrmap e4: x(active, since 2s) 2024-04-26T01:44:54.843 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:54 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3087490793' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T01:44:54.854 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:54.854 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:44:54.854 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:54.856 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T01:44:54.857 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T01:44:54.857 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T01:44:54.858 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:54.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T01:44:54.859 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T01:44:54.867 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T01:44:54.867 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T01:44:54.867 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T01:44:54.867 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T01:44:54.868 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T01:44:24.369478+0000", 2024-04-26T01:44:54.869 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.817+0000 7f4b18200700 1 Processor -- start 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.817+0000 7f4b18200700 1 -- start start 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.818+0000 7f4b18200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1010ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.818+0000 7f4b18200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010b1f0 con 0x7f4b1010a800 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.818+0000 7f4b15800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1010ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.818+0000 7f4b15800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1010ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44122/0 (socket says 172.21.15.37:44122) 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.818+0000 7f4b15800700 1 -- 172.21.15.37:0/1759201108 learned_addr learned my addr 172.21.15.37:0/1759201108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.819+0000 7f4b15800700 1 -- 172.21.15.37:0/1759201108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1010b9b0 con 0x7f4b1010a800 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.819+0000 7f4b15800700 1 --2- 172.21.15.37:0/1759201108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1010ac20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009e80 tx=0x7f4b0400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29babfa2200bf8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:54.870 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.819+0000 7f4b0fe00700 1 -- 172.21.15.37:0/1759201108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b04017070 con 0x7f4b1010a800 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.819+0000 7f4b0fe00700 1 -- 172.21.15.37:0/1759201108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b0400b8c0 con 0x7f4b1010a800 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.819+0000 7f4b0fe00700 1 -- 172.21.15.37:0/1759201108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b04004490 con 0x7f4b1010a800 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.820+0000 7f4b18200700 1 -- 172.21.15.37:0/1759201108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 msgr2=0x7f4b1010ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.820+0000 7f4b18200700 1 --2- 172.21.15.37:0/1759201108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1010ac20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009e80 tx=0x7f4b0400b330 comp rx=0 tx=0).stop 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.820+0000 7f4b18200700 1 -- 172.21.15.37:0/1759201108 shutdown_connections 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.820+0000 7f4b18200700 1 --2- 172.21.15.37:0/1759201108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1010ac20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.820+0000 7f4b18200700 1 -- 172.21.15.37:0/1759201108 >> 172.21.15.37:0/1759201108 conn(0x7f4b1007ddf0 msgr2=0x7f4b101084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.821+0000 7f4b18200700 1 -- 172.21.15.37:0/1759201108 shutdown_connections 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.821+0000 7f4b18200700 1 -- 172.21.15.37:0/1759201108 wait complete. 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b18200700 1 Processor -- start 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b18200700 1 -- start start 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b18200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b18200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1010ebe0 con 0x7f4b1010a800 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b15800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:54.871 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b15800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1019e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44124/0 (socket says 172.21.15.37:44124) 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.822+0000 7f4b15800700 1 -- 172.21.15.37:0/3087490793 learned_addr learned my addr 172.21.15.37:0/3087490793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.823+0000 7f4b15800700 1 -- 172.21.15.37:0/3087490793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1019eb40 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.823+0000 7f4b15800700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1019e600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4b04004a60 tx=0x7f4b0400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.823+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b04020020 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.823+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4b04003c60 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.823+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4b04017040 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.824+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1019edd0 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.824+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101a1ac0 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.825+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1010ed60 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.825+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f4b04023020 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.825+0000 7f4b0e000700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4af8039600 0x7f4af803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.826+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4b0400f050 con 0x7f4b1010a800 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.826+0000 7f4b14e00700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4af8039600 0x7f4af803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.827+0000 7f4b14e00700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4af8039600 0x7f4af803bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:54.872 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:53.833+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b0402a020 con 0x7f4b1010a800 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.160+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4b101a1c90 con 0x7f4b1010a800 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.161+0000 7f4b0e000700 1 -- 172.21.15.37:0/3087490793 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4b0402c300 con 0x7f4b1010a800 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4af8039600 msgr2=0x7f4af803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4af8039600 0x7f4af803bac0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4afc006fd0 tx=0x7f4afc006e40 comp rx=0 tx=0).stop 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 msgr2=0x7f4b1019e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1019e600 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4b04004a60 tx=0x7f4b0400bd50 comp rx=0 tx=0).stop 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 shutdown_connections 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4af8039600 0x7f4af803bac0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 --2- 172.21.15.37:0/3087490793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4b1010a800 0x7f4b1019e600 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 >> 172.21.15.37:0/3087490793 conn(0x7f4b1007ddf0 msgr2=0x7f4b10107b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.165+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 shutdown_connections 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:54.166+0000 7f4b18200700 1 -- 172.21.15.37:0/3087490793 wait complete. 2024-04-26T01:44:54.873 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-04-26T01:44:56.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:55 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3726556613' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T01:44:56.528 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.581+0000 7f46a6800700 1 Processor -- start 2024-04-26T01:44:56.529 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.582+0000 7f46a6800700 1 -- start start 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.582+0000 7f46a6800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a01089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.582+0000 7f46a6800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0108fb0 con 0x7f46a01085c0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.582+0000 7f469f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a01089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.583+0000 7f469f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a01089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44138/0 (socket says 172.21.15.37:44138) 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.583+0000 7f469f400700 1 -- 172.21.15.37:0/1186153856 learned_addr learned my addr 172.21.15.37:0/1186153856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.583+0000 7f469f400700 1 -- 172.21.15.37:0/1186153856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a01097c0 con 0x7f46a01085c0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.584+0000 7f469f400700 1 --2- 172.21.15.37:0/1186153856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a01089e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4690009e80 tx=0x7f469000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c177dfd9c436584f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.584+0000 7f469ea00700 1 -- 172.21.15.37:0/1186153856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4690017070 con 0x7f46a01085c0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.584+0000 7f469ea00700 1 -- 172.21.15.37:0/1186153856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f469000b8c0 con 0x7f46a01085c0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.584+0000 7f469ea00700 1 -- 172.21.15.37:0/1186153856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4690004490 con 0x7f46a01085c0 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 -- 172.21.15.37:0/1186153856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 msgr2=0x7f46a01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 --2- 172.21.15.37:0/1186153856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a01089e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4690009e80 tx=0x7f469000b330 comp rx=0 tx=0).stop 2024-04-26T01:44:56.530 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 -- 172.21.15.37:0/1186153856 shutdown_connections 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 --2- 172.21.15.37:0/1186153856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a01089e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 -- 172.21.15.37:0/1186153856 >> 172.21.15.37:0/1186153856 conn(0x7f46a0103be0 msgr2=0x7f46a0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 -- 172.21.15.37:0/1186153856 shutdown_connections 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.585+0000 7f46a6800700 1 -- 172.21.15.37:0/1186153856 wait complete. 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.586+0000 7f46a6800700 1 Processor -- start 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.586+0000 7f46a6800700 1 -- start start 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.587+0000 7f46a6800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.587+0000 7f46a6800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46a010c9f0 con 0x7f46a01085c0 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.587+0000 7f469f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.587+0000 7f469f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44146/0 (socket says 172.21.15.37:44146) 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.587+0000 7f469f400700 1 -- 172.21.15.37:0/3726556613 learned_addr learned my addr 172.21.15.37:0/3726556613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.587+0000 7f469f400700 1 -- 172.21.15.37:0/3726556613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a019c9a0 con 0x7f46a01085c0 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.588+0000 7f469f400700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a019c460 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f469000b7b0 tx=0x7f469000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.588+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4690020020 con 0x7f46a01085c0 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.588+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4690012430 con 0x7f46a01085c0 2024-04-26T01:44:56.531 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.588+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a019cc30 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.588+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4690017040 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.588+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a019f920 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.589+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f4690023020 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.590+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46a010cb70 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.590+0000 7f469cc00700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4684039560 0x7f468403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.590+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f469000f050 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.590+0000 7f4696a00700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4684039560 0x7f468403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.591+0000 7f4696a00700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4684039560 0x7f468403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4688006fd0 tx=0x7f4688006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.596+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f469002a020 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.828+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f46a00640a0 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.832+0000 7f469cc00700 1 -- 172.21.15.37:0/3726556613 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7f469002d350 con 0x7f46a01085c0 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4684039560 msgr2=0x7f468403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4684039560 0x7f468403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4688006fd0 tx=0x7f4688006e40 comp rx=0 tx=0).stop 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 msgr2=0x7f46a019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:56.532 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a019c460 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f469000b7b0 tx=0x7f469000be20 comp rx=0 tx=0).stop 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 shutdown_connections 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f4684039560 0x7f468403ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 --2- 172.21.15.37:0/3726556613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f46a01085c0 0x7f46a019c460 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.836+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 >> 172.21.15.37:0/3726556613 conn(0x7f46a0103be0 msgr2=0x7f46a0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.837+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 shutdown_connections 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:55.837+0000 7f46a6800700 1 -- 172.21.15.37:0/3726556613 wait complete. 2024-04-26T01:44:56.533 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-04-26T01:44:57.815 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:57 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3009572111' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T01:44:57.815 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:57 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: ignoring --setuser ceph since I am not root 2024-04-26T01:44:57.815 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:57 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: ignoring --setgroup ceph since I am not root 2024-04-26T01:44:57.815 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:57 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:57.817+0000 7f76442e0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:44:58.071 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:58 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:58.073+0000 7f76442e0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:44:58.283 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.162+0000 7ff9a5800700 1 Processor -- start 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.163+0000 7ff9a5800700 1 -- start start 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.163+0000 7ff9a5800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a007b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.163+0000 7ff9a5800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a007b780 con 0x7ff9a007cd50 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.163+0000 7ff99ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a007b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.163+0000 7ff99ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a007b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44148/0 (socket says 172.21.15.37:44148) 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.163+0000 7ff99ea00700 1 -- 172.21.15.37:0/2464085775 learned_addr learned my addr 172.21.15.37:0/2464085775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.164+0000 7ff99ea00700 1 -- 172.21.15.37:0/2464085775 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a007b900 con 0x7ff9a007cd50 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.164+0000 7ff99ea00700 1 --2- 172.21.15.37:0/2464085775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a007b1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9900098a0 tx=0x7ff990009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4822b8b6a02a25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.165+0000 7ff99e000700 1 -- 172.21.15.37:0/2464085775 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990004160 con 0x7ff9a007cd50 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.165+0000 7ff99e000700 1 -- 172.21.15.37:0/2464085775 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff990004300 con 0x7ff9a007cd50 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.165+0000 7ff99e000700 1 -- 172.21.15.37:0/2464085775 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990011af0 con 0x7ff9a007cd50 2024-04-26T01:44:58.284 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.165+0000 7ff9a5800700 1 -- 172.21.15.37:0/2464085775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 msgr2=0x7ff9a007b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.165+0000 7ff9a5800700 1 --2- 172.21.15.37:0/2464085775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a007b1b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff9900098a0 tx=0x7ff990009bb0 comp rx=0 tx=0).stop 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.166+0000 7ff9a5800700 1 -- 172.21.15.37:0/2464085775 shutdown_connections 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.166+0000 7ff9a5800700 1 --2- 172.21.15.37:0/2464085775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a007b1b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.166+0000 7ff9a5800700 1 -- 172.21.15.37:0/2464085775 >> 172.21.15.37:0/2464085775 conn(0x7ff9a0103be0 msgr2=0x7ff9a0106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.166+0000 7ff9a5800700 1 -- 172.21.15.37:0/2464085775 shutdown_connections 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.166+0000 7ff9a5800700 1 -- 172.21.15.37:0/2464085775 wait complete. 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.167+0000 7ff9a5800700 1 Processor -- start 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.167+0000 7ff9a5800700 1 -- start start 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.167+0000 7ff9a5800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a019e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.167+0000 7ff9a5800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a010eb30 con 0x7ff9a007cd50 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.167+0000 7ff99ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a019e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.168+0000 7ff99ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a019e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44160/0 (socket says 172.21.15.37:44160) 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.168+0000 7ff99ea00700 1 -- 172.21.15.37:0/3009572111 learned_addr learned my addr 172.21.15.37:0/3009572111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:58.285 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.168+0000 7ff99ea00700 1 -- 172.21.15.37:0/3009572111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a019ec20 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.168+0000 7ff99ea00700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a019e6e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff990011ac0 tx=0x7ff990011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.169+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990003970 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.169+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff990003b10 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.169+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff990024610 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.169+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a019eeb0 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.169+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01a1ba0 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.170+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a00640a0 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.170+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7ff99001a9e0 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.171+0000 7ff997400700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff9840395b0 0x7ff98403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.171+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff99004d860 con 0x7ff9a007cd50 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.171+0000 7ff997e00700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff9840395b0 0x7ff98403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:58.286 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.172+0000 7ff997e00700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff9840395b0 0x7ff98403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff988006fd0 tx=0x7ff988006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.176+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff99000b3e0 con 0x7ff9a007cd50 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.469+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7ff9a01a1d70 con 0x7ff9a007cd50 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.522+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7ff99004d280 con 0x7ff9a007cd50 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.525+0000 7ff997400700 1 -- 172.21.15.37:0/3009572111 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff99002c350 con 0x7ff9a007cd50 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.528+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff9840395b0 msgr2=0x7ff98403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.528+0000 7ff9a5800700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff9840395b0 0x7ff98403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff988006fd0 tx=0x7ff988006e40 comp rx=0 tx=0).stop 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.528+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 msgr2=0x7ff9a019e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.528+0000 7ff9a5800700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a019e6e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff990011ac0 tx=0x7ff990011ba0 comp rx=0 tx=0).stop 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.529+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 shutdown_connections 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.529+0000 7ff9a5800700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff9840395b0 0x7ff98403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.529+0000 7ff9a5800700 1 --2- 172.21.15.37:0/3009572111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff9a007cd50 0x7ff9a019e6e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.529+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 >> 172.21.15.37:0/3009572111 conn(0x7ff9a0103be0 msgr2=0x7ff9a0104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.529+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 shutdown_connections 2024-04-26T01:44:58.287 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:57.529+0000 7ff9a5800700 1 -- 172.21.15.37:0/3009572111 wait complete. 2024-04-26T01:44:58.433 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:58 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:58.209+0000 7f76442e0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:44:58.706 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:58 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:58.625+0000 7f76442e0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:44:58.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:58 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3009572111' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T01:44:58.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:58 smithi037 ceph-mon[27906]: mgrmap e5: x(active, since 5s) 2024-04-26T01:44:59.161 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:44:58 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:44:58.744+0000 7f76442e0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:44:59.903 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:44:59 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3177468713' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T01:44:59.912 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:44:59.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T01:44:59.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:44:59.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T01:44:59.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T01:44:59.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:44:59.913 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.953+0000 7f75a2000700 1 Processor -- start 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.953+0000 7f75a2000700 1 -- start start 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.954+0000 7f75a2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c10b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.954+0000 7f75a2000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f759c063c80 con 0x7f759c109120 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.954+0000 7f759b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c10b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.954+0000 7f759b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c10b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44174/0 (socket says 172.21.15.37:44174) 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.954+0000 7f759b400700 1 -- 172.21.15.37:0/3247330458 learned_addr learned my addr 172.21.15.37:0/3247330458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.955+0000 7f759b400700 1 -- 172.21.15.37:0/3247330458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c10ba50 con 0x7f759c109120 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.955+0000 7f759b400700 1 --2- 172.21.15.37:0/3247330458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c10b510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098a0 tx=0x7f758c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0c42d27fc0e019 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.955+0000 7f759a000700 1 -- 172.21.15.37:0/3247330458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c004160 con 0x7f759c109120 2024-04-26T01:44:59.916 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.955+0000 7f759a000700 1 -- 172.21.15.37:0/3247330458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f758c004300 con 0x7f759c109120 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.955+0000 7f759a000700 1 -- 172.21.15.37:0/3247330458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c011af0 con 0x7f759c109120 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.956+0000 7f75a2000700 1 -- 172.21.15.37:0/3247330458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 msgr2=0x7f759c10b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.956+0000 7f75a2000700 1 --2- 172.21.15.37:0/3247330458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c10b510 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098a0 tx=0x7f758c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.956+0000 7f75a2000700 1 -- 172.21.15.37:0/3247330458 shutdown_connections 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.956+0000 7f75a2000700 1 --2- 172.21.15.37:0/3247330458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c10b510 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.956+0000 7f75a2000700 1 -- 172.21.15.37:0/3247330458 >> 172.21.15.37:0/3247330458 conn(0x7f759c102ab0 msgr2=0x7f759c104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.957+0000 7f75a2000700 1 -- 172.21.15.37:0/3247330458 shutdown_connections 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.957+0000 7f75a2000700 1 -- 172.21.15.37:0/3247330458 wait complete. 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.958+0000 7f75a2000700 1 Processor -- start 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.958+0000 7f75a2000700 1 -- start start 2024-04-26T01:44:59.917 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.958+0000 7f75a2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.958+0000 7f75a2000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f759c10c8e0 con 0x7f759c109120 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.959+0000 7f759b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.959+0000 7f759b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44176/0 (socket says 172.21.15.37:44176) 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.959+0000 7f759b400700 1 -- 172.21.15.37:0/3177468713 learned_addr learned my addr 172.21.15.37:0/3177468713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.959+0000 7f759b400700 1 -- 172.21.15.37:0/3177468713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c19ebc0 con 0x7f759c109120 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.959+0000 7f759b400700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c19e680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f758c011ac0 tx=0x7f758c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.959+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c003b20 con 0x7f759c109120 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.960+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f758c01a8c0 con 0x7f759c109120 2024-04-26T01:44:59.921 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.960+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c0248d0 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.960+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c19ee50 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.960+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c1a1b40 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.961+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f758c022020 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.961+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c0640a0 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.961+0000 7f7593e00700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 0x7f758003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.962+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f758c04d5c0 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.962+0000 7f759aa00700 1 -- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 msgr2=0x7f758003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.962+0000 7f759aa00700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 0x7f758003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:58.967+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f758c02cdf0 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.163+0000 7f759aa00700 1 -- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 msgr2=0x7f758003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.163+0000 7f759aa00700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 0x7f758003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.259+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f759c1a1d10 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.260+0000 7f7593e00700 1 -- 172.21.15.37:0/3177468713 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f758c00b5a0 con 0x7f759c109120 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 msgr2=0x7f758003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:44:59.922 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 0x7f758003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 msgr2=0x7f759c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c19e680 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f758c011ac0 tx=0x7f758c011ba0 comp rx=0 tx=0).stop 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 shutdown_connections 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7f75800395b0 0x7f758003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.263+0000 7f75a2000700 1 --2- 172.21.15.37:0/3177468713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f759c109120 0x7f759c19e680 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.264+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 >> 172.21.15.37:0/3177468713 conn(0x7f759c102ab0 msgr2=0x7f759c1035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.264+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 shutdown_connections 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:44:59.264+0000 7f75a2000700 1 -- 172.21.15.37:0/3177468713 wait complete. 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-26T01:44:59.923 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-04-26T01:45:00.447 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:00.367+0000 7f76442e0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:45:00.706 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:00.604+0000 7f76442e0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:45:01.117 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:00.824+0000 7f76442e0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:45:01.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:01.217+0000 7f76442e0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:45:01.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:01.343+0000 7f76442e0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:45:01.861 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:01.600+0000 7f76442e0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:45:01.861 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:01.863+0000 7f76442e0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:45:02.957 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:02 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:02.673+0000 7f76442e0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:45:03.520 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:03.243+0000 7f76442e0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:45:03.520 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:03.376+0000 7f76442e0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:45:03.957 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:03.665+0000 7f76442e0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:45:04.706 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:04.349+0000 7f76442e0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:45:04.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:04.476+0000 7f76442e0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:45:05.120 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:04.808+0000 7f76442e0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:45:05.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:04.993+0000 7f76442e0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:45:05.121 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:05.123+0000 7f76442e0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:45:05.456 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:05.279+0000 7f76442e0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:45:05.956 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:05.653+0000 7f76442e0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:45:06.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:06.438+0000 7f76442e0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:06.736+0000 7f76442e0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: Active manager daemon x restarted 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: Activating manager daemon x 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: mgrmap e6: x(active, starting, since 0.0074401s) 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:45:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:45:07.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:45:07.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:45:07.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:45:07.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:06 smithi037 ceph-mon[27906]: Manager daemon x is now available 2024-04-26T01:45:08.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: Found migration_current of "None". Setting to last migration. 2024-04-26T01:45:08.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:08.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:08.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:08.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:08.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:45:08.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:45:08.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:08.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:08.114 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:07 smithi037 ceph-mon[27906]: mgrmap e7: x(active, since 1.0151s) 2024-04-26T01:45:08.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.510+0000 7ff8da400700 1 Processor -- start 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.510+0000 7ff8da400700 1 -- start start 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.510+0000 7ff8da400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d41089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.511+0000 7ff8da400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d4108fb0 con 0x7ff8d41085c0 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.511+0000 7ff8d3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d41089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.511+0000 7ff8d3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d41089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36412/0 (socket says 172.21.15.37:36412) 2024-04-26T01:45:08.478 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.511+0000 7ff8d3400700 1 -- 172.21.15.37:0/3031948103 learned_addr learned my addr 172.21.15.37:0/3031948103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.511+0000 7ff8d3400700 1 -- 172.21.15.37:0/3031948103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d41097c0 con 0x7ff8d41085c0 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.512+0000 7ff8d3400700 1 --2- 172.21.15.37:0/3031948103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d41089e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40098a0 tx=0x7ff8c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361f30aa9aec0b99 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.512+0000 7ff8d2000700 1 -- 172.21.15.37:0/3031948103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4004160 con 0x7ff8d41085c0 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.512+0000 7ff8d2000700 1 -- 172.21.15.37:0/3031948103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff8c4004300 con 0x7ff8d41085c0 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.512+0000 7ff8d2000700 1 -- 172.21.15.37:0/3031948103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4011af0 con 0x7ff8d41085c0 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 -- 172.21.15.37:0/3031948103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 msgr2=0x7ff8d41089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 --2- 172.21.15.37:0/3031948103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d41089e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40098a0 tx=0x7ff8c4009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 -- 172.21.15.37:0/3031948103 shutdown_connections 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 --2- 172.21.15.37:0/3031948103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d41089e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 -- 172.21.15.37:0/3031948103 >> 172.21.15.37:0/3031948103 conn(0x7ff8d4103be0 msgr2=0x7ff8d4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 -- 172.21.15.37:0/3031948103 shutdown_connections 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.513+0000 7ff8da400700 1 -- 172.21.15.37:0/3031948103 wait complete. 2024-04-26T01:45:08.479 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.514+0000 7ff8da400700 1 Processor -- start 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.514+0000 7ff8da400700 1 -- start start 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.515+0000 7ff8da400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d407ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.515+0000 7ff8da400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d410c9f0 con 0x7ff8d41085c0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.515+0000 7ff8d3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d407ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.515+0000 7ff8d3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d407ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36416/0 (socket says 172.21.15.37:36416) 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.515+0000 7ff8d3400700 1 -- 172.21.15.37:0/1958193617 learned_addr learned my addr 172.21.15.37:0/1958193617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.515+0000 7ff8d3400700 1 -- 172.21.15.37:0/1958193617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d407cf80 con 0x7ff8d41085c0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.516+0000 7ff8d3400700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d407ca40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40117b0 tx=0x7ff8c4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.516+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4003af0 con 0x7ff8d41085c0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.516+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff8c4020070 con 0x7ff8d41085c0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.516+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8c4019630 con 0x7ff8d41085c0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.516+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d407b170 con 0x7ff8d41085c0 2024-04-26T01:45:08.480 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.516+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d407b6b0 con 0x7ff8d41085c0 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.517+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7ff8c401a9c0 con 0x7ff8d41085c0 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.518+0000 7ff8cbe00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.518+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 --> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8b803c230 con 0x7ff8b8039660 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.518+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8c404db20 con 0x7ff8d41085c0 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.518+0000 7ff8d2a00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.518+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.719+0000 7ff8d2a00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:00.719+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:01.120+0000 7ff8d2a00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:01.120+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:01.921+0000 7ff8d2a00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:01.921+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T01:45:08.481 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:03.523+0000 7ff8d2a00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:03.523+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:06.726+0000 7ff8d2a00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/1210493699 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:06.726+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:06.746+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7ff8c40198a0 con 0x7ff8d41085c0 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:06.746+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 msgr2=0x7ff8b803bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:06.746+0000 7ff8cbe00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.751+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7ff8c401a430 con 0x7ff8d41085c0 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.751+0000 7ff8cbe00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff8b803cf50 0x7ff8b803f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.751+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff8c404d230 con 0x7ff8b803cf50 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.753+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff8b803cf50 0x7ff8b803f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.757+0000 7ff8d2a00700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff8b803cf50 0x7ff8b803f340 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0003a10 tx=0x7ff8c00092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.758+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7ff8c404d230 con 0x7ff8b803cf50 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.766+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7ff8d410cb70 con 0x7ff8b803cf50 2024-04-26T01:45:08.482 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.767+0000 7ff8cbe00700 1 -- 172.21.15.37:0/1958193617 <== mgr.14120 v2:172.21.15.37:6800/2267935573 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7ff8d410cb70 con 0x7ff8b803cf50 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.767+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff8b803cf50 msgr2=0x7ff8b803f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.767+0000 7ff8da400700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff8b803cf50 0x7ff8b803f340 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0003a10 tx=0x7ff8c00092b0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.767+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 msgr2=0x7ff8d407ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.767+0000 7ff8da400700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d407ca40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff8c40117b0 tx=0x7ff8c4011b70 comp rx=0 tx=0).stop 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 shutdown_connections 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff8b803cf50 0x7ff8b803f340 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8d41085c0 0x7ff8d407ca40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 --2- 172.21.15.37:0/1958193617 >> [v2:172.21.15.37:6800/1210493699,v1:172.21.15.37:6801/1210493699] conn(0x7ff8b8039660 0x7ff8b803bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 >> 172.21.15.37:0/1958193617 conn(0x7ff8d4103be0 msgr2=0x7ff8d4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 shutdown_connections 2024-04-26T01:45:08.483 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:07.768+0000 7ff8da400700 1 -- 172.21.15.37:0/1958193617 wait complete. 2024-04-26T01:45:08.484 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-04-26T01:45:08.484 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:07] ENGINE Bus STARTING 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:07] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:07] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:07] ENGINE Bus STARTED 2024-04-26T01:45:09.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:08 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:10.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:09 smithi037 ceph-mon[27906]: mgrmap e8: x(active, since 2s) 2024-04-26T01:45:10.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:09 smithi037 ceph-mon[27906]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:10.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:09 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:10.107 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:09 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:10.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.126+0000 7f8836000700 1 Processor -- start 2024-04-26T01:45:10.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.127+0000 7f8836000700 1 -- start start 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.127+0000 7f8836000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883010aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.127+0000 7f8836000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f883010b010 con 0x7f883010a620 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.127+0000 7f8834c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883010aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.128+0000 7f8834c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883010aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36494/0 (socket says 172.21.15.37:36494) 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.128+0000 7f8834c00700 1 -- 172.21.15.37:0/3776143688 learned_addr learned my addr 172.21.15.37:0/3776143688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.128+0000 7f8834c00700 1 -- 172.21.15.37:0/3776143688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883010b7d0 con 0x7f883010a620 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.128+0000 7f8834c00700 1 --2- 172.21.15.37:0/3776143688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883010aa40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88200098a0 tx=0x7f8820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d097bfb2f2f96932 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.129+0000 7f882fe00700 1 -- 172.21.15.37:0/3776143688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820004160 con 0x7f883010a620 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.129+0000 7f882fe00700 1 -- 172.21.15.37:0/3776143688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8820004300 con 0x7f883010a620 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.129+0000 7f882fe00700 1 -- 172.21.15.37:0/3776143688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820011af0 con 0x7f883010a620 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.129+0000 7f8836000700 1 -- 172.21.15.37:0/3776143688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 msgr2=0x7f883010aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.129+0000 7f8836000700 1 --2- 172.21.15.37:0/3776143688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883010aa40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88200098a0 tx=0x7f8820009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:10.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.130+0000 7f8836000700 1 -- 172.21.15.37:0/3776143688 shutdown_connections 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.130+0000 7f8836000700 1 --2- 172.21.15.37:0/3776143688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883010aa40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.130+0000 7f8836000700 1 -- 172.21.15.37:0/3776143688 >> 172.21.15.37:0/3776143688 conn(0x7f883007db40 msgr2=0x7f88301084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.130+0000 7f8836000700 1 -- 172.21.15.37:0/3776143688 shutdown_connections 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.130+0000 7f8836000700 1 -- 172.21.15.37:0/3776143688 wait complete. 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.131+0000 7f8836000700 1 Processor -- start 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.131+0000 7f8836000700 1 -- start start 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.131+0000 7f8836000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883019e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.131+0000 7f8836000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f883010ea00 con 0x7f883010a620 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.132+0000 7f8834c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883019e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.132+0000 7f8834c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883019e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36496/0 (socket says 172.21.15.37:36496) 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.132+0000 7f8834c00700 1 -- 172.21.15.37:0/2106978780 learned_addr learned my addr 172.21.15.37:0/2106978780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.132+0000 7f8834c00700 1 -- 172.21.15.37:0/2106978780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f883019ea00 con 0x7f883010a620 2024-04-26T01:45:10.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.132+0000 7f8834c00700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883019e4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f8820011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.133+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820003b20 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.133+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f882001a8c0 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.133+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8820024940 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.133+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f883019ec90 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.133+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88301a1980 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.134+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8820022020 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.134+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883010eb80 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.134+0000 7f882e000700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f8814039660 0x7f881403bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.135+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f882004d8a0 con 0x7f883010a620 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.135+0000 7f8827e00700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f8814039660 0x7f881403bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.136+0000 7f8827e00700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f8814039660 0x7f881403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f881c006fd0 tx=0x7f881c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:10.127 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.142+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f882000be50 con 0x7f883010a620 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.390+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f88301082e0 con 0x7f8814039660 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.403+0000 7f882e000700 1 -- 172.21.15.37:0/2106978780 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f88301082e0 con 0x7f8814039660 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f8814039660 msgr2=0x7f881403bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f8814039660 0x7f881403bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f881c006fd0 tx=0x7f881c006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 msgr2=0x7f883019e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883019e4c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8820000c00 tx=0x7f8820011b70 comp rx=0 tx=0).stop 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 shutdown_connections 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f8814039660 0x7f881403bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 --2- 172.21.15.37:0/2106978780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f883010a620 0x7f883019e4c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.406+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 >> 172.21.15.37:0/2106978780 conn(0x7f883007db40 msgr2=0x7f8830107bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.407+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 shutdown_connections 2024-04-26T01:45:10.128 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:09.407+0000 7f8836000700 1 -- 172.21.15.37:0/2106978780 wait complete. 2024-04-26T01:45:11.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T01:45:11.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.758+0000 7f20a4600700 1 Processor -- start 2024-04-26T01:45:11.881 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.758+0000 7f20a4600700 1 -- start start 2024-04-26T01:45:11.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.759+0000 7f20a4600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c10f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:11.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.759+0000 7f20a4600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f209c10fa30 con 0x7f209c10f040 2024-04-26T01:45:11.882 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.759+0000 7f20a1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c10f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:11.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.759+0000 7f20a1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c10f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60364/0 (socket says 172.21.15.37:60364) 2024-04-26T01:45:11.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.759+0000 7f20a1c00700 1 -- 172.21.15.37:0/3387554244 learned_addr learned my addr 172.21.15.37:0/3387554244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:11.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.759+0000 7f20a1c00700 1 -- 172.21.15.37:0/3387554244 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c10fdc0 con 0x7f209c10f040 2024-04-26T01:45:11.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.760+0000 7f20a1c00700 1 --2- 172.21.15.37:0/3387554244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c10f460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f20900098a0 tx=0x7f2090009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234d06131e602200 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:11.885 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.760+0000 7f209be00700 1 -- 172.21.15.37:0/3387554244 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2090004160 con 0x7f209c10f040 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.760+0000 7f209be00700 1 -- 172.21.15.37:0/3387554244 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2090004300 con 0x7f209c10f040 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.760+0000 7f209be00700 1 -- 172.21.15.37:0/3387554244 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2090011af0 con 0x7f209c10f040 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 -- 172.21.15.37:0/3387554244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 msgr2=0x7f209c10f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 --2- 172.21.15.37:0/3387554244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c10f460 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f20900098a0 tx=0x7f2090009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 -- 172.21.15.37:0/3387554244 shutdown_connections 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 --2- 172.21.15.37:0/3387554244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c10f460 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 -- 172.21.15.37:0/3387554244 >> 172.21.15.37:0/3387554244 conn(0x7f209c07ddf0 msgr2=0x7f209c080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 -- 172.21.15.37:0/3387554244 shutdown_connections 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.761+0000 7f20a4600700 1 -- 172.21.15.37:0/3387554244 wait complete. 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.762+0000 7f20a4600700 1 Processor -- start 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.762+0000 7f20a4600700 1 -- start start 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.763+0000 7f20a4600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c1a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:11.886 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.763+0000 7f20a4600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f209c112ff0 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.763+0000 7f20a1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c1a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.763+0000 7f20a1c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c1a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60378/0 (socket says 172.21.15.37:60378) 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.763+0000 7f20a1c00700 1 -- 172.21.15.37:0/3702167836 learned_addr learned my addr 172.21.15.37:0/3702167836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.763+0000 7f20a1c00700 1 -- 172.21.15.37:0/3702167836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c1a2fa0 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.764+0000 7f20a1c00700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c1a2a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2090003f40 tx=0x7f2090011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.764+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20900038b0 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.764+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2090003a50 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.764+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f209001a5c0 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.764+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c1a3230 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.764+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c1a5f20 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.765+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2090024430 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.765+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c113170 con 0x7f209c10f040 2024-04-26T01:45:11.887 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.766+0000 7f209a000700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f20840395b0 0x7f208403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:11.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.766+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f209004db30 con 0x7f209c10f040 2024-04-26T01:45:11.889 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.766+0000 7f20a1200700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f20840395b0 0x7f208403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.767+0000 7f20a1200700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f20840395b0 0x7f208403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2088006fd0 tx=0x7f2088006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:10.772+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f209004d480 con 0x7f209c10f040 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.003+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f209c07ffb0 con 0x7f20840395b0 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.005+0000 7f209a000700 1 -- 172.21.15.37:0/3702167836 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f209c07ffb0 con 0x7f20840395b0 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f20840395b0 msgr2=0x7f208403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f20840395b0 0x7f208403ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2088006fd0 tx=0x7f2088006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 msgr2=0x7f209c1a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c1a2a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2090003f40 tx=0x7f2090011890 comp rx=0 tx=0).stop 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 shutdown_connections 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f20840395b0 0x7f208403ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 --2- 172.21.15.37:0/3702167836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f209c10f040 0x7f209c1a2a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 >> 172.21.15.37:0/3702167836 conn(0x7f209c07ddf0 msgr2=0x7f209c07f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:11.890 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 shutdown_connections 2024-04-26T01:45:11.891 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:11.008+0000 7f20a4600700 1 -- 172.21.15.37:0/3702167836 wait complete. 2024-04-26T01:45:11.891 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-04-26T01:45:11.956 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:11 smithi037 ceph-mon[27906]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:13.458 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: Generating public/private rsa key pair. 2024-04-26T01:45:13.458 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: Your identification has been saved in /tmp/tmpu_gg1_ap/key. 2024-04-26T01:45:13.458 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: Your public key has been saved in /tmp/tmpu_gg1_ap/key.pub. 2024-04-26T01:45:13.458 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: The key fingerprint is: 2024-04-26T01:45:13.458 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: SHA256:g0muS0d0vbVeHZp/KiEu9+7N4190IT5/e4AOqVY4PVA ceph-5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: The key's randomart image is: 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: +---[RSA 3072]----+ 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | | 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | .E | 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | o ... .. o | 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | + +. o..+.o| 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | = S+...*. +| 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | o o.B.o.=..| 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | o . = =.. ++| 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | . o + o oo..*| 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: | . . o ++o=+o| 2024-04-26T01:45:13.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: +----[SHA256]-----+ 2024-04-26T01:45:13.830 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.492+0000 7f7e78e00700 1 Processor -- start 2024-04-26T01:45:13.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.492+0000 7f7e78e00700 1 -- start start 2024-04-26T01:45:13.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.493+0000 7f7e78e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e741089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:13.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.493+0000 7f7e78e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e74108fb0 con 0x7f7e741085c0 2024-04-26T01:45:13.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.493+0000 7f7e71e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e741089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:13.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.494+0000 7f7e71e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e741089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60382/0 (socket says 172.21.15.37:60382) 2024-04-26T01:45:13.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.494+0000 7f7e71e00700 1 -- 172.21.15.37:0/2409737903 learned_addr learned my addr 172.21.15.37:0/2409737903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:13.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.494+0000 7f7e71e00700 1 -- 172.21.15.37:0/2409737903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e741097c0 con 0x7f7e741085c0 2024-04-26T01:45:13.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.494+0000 7f7e71e00700 1 --2- 172.21.15.37:0/2409737903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e741089e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e640098a0 tx=0x7f7e64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9dfe265cb97c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:13.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.495+0000 7f7e70a00700 1 -- 172.21.15.37:0/2409737903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64004160 con 0x7f7e741085c0 2024-04-26T01:45:13.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.495+0000 7f7e70a00700 1 -- 172.21.15.37:0/2409737903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e64004300 con 0x7f7e741085c0 2024-04-26T01:45:13.837 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.495+0000 7f7e70a00700 1 -- 172.21.15.37:0/2409737903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64011af0 con 0x7f7e741085c0 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.495+0000 7f7e78e00700 1 -- 172.21.15.37:0/2409737903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 msgr2=0x7f7e741089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.495+0000 7f7e78e00700 1 --2- 172.21.15.37:0/2409737903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e741089e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7e640098a0 tx=0x7f7e64009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.496+0000 7f7e78e00700 1 -- 172.21.15.37:0/2409737903 shutdown_connections 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.496+0000 7f7e78e00700 1 --2- 172.21.15.37:0/2409737903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e741089e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.496+0000 7f7e78e00700 1 -- 172.21.15.37:0/2409737903 >> 172.21.15.37:0/2409737903 conn(0x7f7e74103be0 msgr2=0x7f7e74106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.496+0000 7f7e78e00700 1 -- 172.21.15.37:0/2409737903 shutdown_connections 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.496+0000 7f7e78e00700 1 -- 172.21.15.37:0/2409737903 wait complete. 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.497+0000 7f7e78e00700 1 Processor -- start 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.498+0000 7f7e78e00700 1 -- start start 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.498+0000 7f7e78e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e7419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:13.838 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.498+0000 7f7e78e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7e7410c9f0 con 0x7f7e741085c0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.498+0000 7f7e71e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e7419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.498+0000 7f7e71e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e7419e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60394/0 (socket says 172.21.15.37:60394) 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.498+0000 7f7e71e00700 1 -- 172.21.15.37:0/575136706 learned_addr learned my addr 172.21.15.37:0/575136706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e71e00700 1 -- 172.21.15.37:0/575136706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e7419ec90 con 0x7f7e741085c0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e71e00700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e7419e750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e640117b0 tx=0x7f7e64011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64003af0 con 0x7f7e741085c0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e64020070 con 0x7f7e741085c0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e64019630 con 0x7f7e741085c0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e7419ef20 con 0x7f7e741085c0 2024-04-26T01:45:13.839 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.499+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e741a1c10 con 0x7f7e741085c0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.501+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7e6401a9c0 con 0x7f7e741085c0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.500+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e7410cb70 con 0x7f7e741085c0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.501+0000 7f7e6aa00700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7e58039600 0x7f7e5803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.501+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e6404db30 con 0x7f7e741085c0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.501+0000 7f7e71400700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7e58039600 0x7f7e5803bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.502+0000 7f7e71400700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7e58039600 0x7f7e5803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.507+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e640042a0 con 0x7f7e741085c0 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:12.759+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f7e74104e10 con 0x7f7e58039600 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.119+0000 7f7e6aa00700 1 -- 172.21.15.37:0/575136706 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7e74104e10 con 0x7f7e58039600 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7e58039600 msgr2=0x7f7e5803bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7e58039600 0x7f7e5803bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7e60006fd0 tx=0x7f7e60006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:13.840 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 msgr2=0x7f7e7419e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e7419e750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e640117b0 tx=0x7f7e64011b70 comp rx=0 tx=0).stop 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 shutdown_connections 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7e58039600 0x7f7e5803bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 --2- 172.21.15.37:0/575136706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7e741085c0 0x7f7e7419e750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.122+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 >> 172.21.15.37:0/575136706 conn(0x7f7e74103be0 msgr2=0x7f7e74104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.123+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 shutdown_connections 2024-04-26T01:45:13.841 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:13.123+0000 7f7e78e00700 1 -- 172.21.15.37:0/575136706 wait complete. 2024-04-26T01:45:14.413 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:14 smithi037 ceph-mon[27906]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:14.413 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:14 smithi037 ceph-mon[27906]: Generating ssh key... 2024-04-26T01:45:14.413 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:14 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:14.413 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:14 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:15.370 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:45:15.370 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.493+0000 7f7a52e00700 1 Processor -- start 2024-04-26T01:45:15.370 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.494+0000 7f7a52e00700 1 -- start start 2024-04-26T01:45:15.370 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.494+0000 7f7a52e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c10ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.494+0000 7f7a52e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c10b1f0 con 0x7f7a4c10a800 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.495+0000 7f7a4be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c10ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.495+0000 7f7a4be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c10ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60400/0 (socket says 172.21.15.37:60400) 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.495+0000 7f7a4be00700 1 -- 172.21.15.37:0/4132983596 learned_addr learned my addr 172.21.15.37:0/4132983596 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.495+0000 7f7a4be00700 1 -- 172.21.15.37:0/4132983596 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c10b9b0 con 0x7f7a4c10a800 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.495+0000 7f7a4be00700 1 --2- 172.21.15.37:0/4132983596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c10ac20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0098a0 tx=0x7f7a3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e2dc89727240d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.496+0000 7f7a4aa00700 1 -- 172.21.15.37:0/4132983596 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c004160 con 0x7f7a4c10a800 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.496+0000 7f7a4aa00700 1 -- 172.21.15.37:0/4132983596 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a3c004300 con 0x7f7a4c10a800 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.496+0000 7f7a4aa00700 1 -- 172.21.15.37:0/4132983596 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c011af0 con 0x7f7a4c10a800 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.496+0000 7f7a52e00700 1 -- 172.21.15.37:0/4132983596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 msgr2=0x7f7a4c10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.496+0000 7f7a52e00700 1 --2- 172.21.15.37:0/4132983596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c10ac20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c0098a0 tx=0x7f7a3c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.497+0000 7f7a52e00700 1 -- 172.21.15.37:0/4132983596 shutdown_connections 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.497+0000 7f7a52e00700 1 --2- 172.21.15.37:0/4132983596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c10ac20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:15.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.497+0000 7f7a52e00700 1 -- 172.21.15.37:0/4132983596 >> 172.21.15.37:0/4132983596 conn(0x7f7a4c07ddf0 msgr2=0x7f7a4c1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.497+0000 7f7a52e00700 1 -- 172.21.15.37:0/4132983596 shutdown_connections 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.497+0000 7f7a52e00700 1 -- 172.21.15.37:0/4132983596 wait complete. 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.498+0000 7f7a52e00700 1 Processor -- start 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.498+0000 7f7a52e00700 1 -- start start 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.499+0000 7f7a52e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.499+0000 7f7a52e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a4c10ebe0 con 0x7f7a4c10a800 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.499+0000 7f7a4be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.499+0000 7f7a4be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60406/0 (socket says 172.21.15.37:60406) 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.499+0000 7f7a4be00700 1 -- 172.21.15.37:0/2991723443 learned_addr learned my addr 172.21.15.37:0/2991723443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.499+0000 7f7a4be00700 1 -- 172.21.15.37:0/2991723443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a4c19ebc0 con 0x7f7a4c10a800 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.500+0000 7f7a4be00700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c19e680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c000c00 tx=0x7f7a3c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.500+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c003b20 con 0x7f7a4c10a800 2024-04-26T01:45:15.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.500+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7a3c01a8c0 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.500+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a3c024940 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.500+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a4c19ee50 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.500+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a4c1a1b40 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.501+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a4c10ed60 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.502+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7a3c022020 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.502+0000 7f7a48c00700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7a30039600 0x7f7a3003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.502+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7a3c04d8a0 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.503+0000 7f7a4b400700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7a30039600 0x7f7a3003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.503+0000 7f7a4b400700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7a30039600 0x7f7a3003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a38006fd0 tx=0x7f7a38006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.508+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7a3c02c3a0 con 0x7f7a4c10a800 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.739+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7a4c1082c0 con 0x7f7a30039600 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.741+0000 7f7a48c00700 1 -- 172.21.15.37:0/2991723443 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7a4c1082c0 con 0x7f7a30039600 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7a30039600 msgr2=0x7f7a3003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:15.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7a30039600 0x7f7a3003bac0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7a38006fd0 tx=0x7f7a38006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 msgr2=0x7f7a4c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c19e680 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c000c00 tx=0x7f7a3c011b70 comp rx=0 tx=0).stop 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 shutdown_connections 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f7a30039600 0x7f7a3003bac0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 --2- 172.21.15.37:0/2991723443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a4c10a800 0x7f7a4c19e680 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 >> 172.21.15.37:0/2991723443 conn(0x7f7a4c07ddf0 msgr2=0x7f7a4c107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 shutdown_connections 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:14.745+0000 7f7a52e00700 1 -- 172.21.15.37:0/2991723443 wait complete. 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T01:45:15.374 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T01:45:15.378 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-04-26T01:45:15.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:15 smithi037 ceph-mon[27906]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:16.898 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:16 smithi037 ceph-mon[27906]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:17.897 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:17 smithi037 ceph-mon[27906]: Deploying cephadm binary to smithi037 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.044+0000 7ff72c400700 1 Processor -- start 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.044+0000 7ff72c400700 1 -- start start 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.045+0000 7ff72c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff72410f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.045+0000 7ff72c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff72410fa30 con 0x7ff72410f040 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.045+0000 7ff729a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff72410f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.045+0000 7ff729a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff72410f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60414/0 (socket says 172.21.15.37:60414) 2024-04-26T01:45:23.383 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.045+0000 7ff729a00700 1 -- 172.21.15.37:0/1858392172 learned_addr learned my addr 172.21.15.37:0/1858392172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.046+0000 7ff729a00700 1 -- 172.21.15.37:0/1858392172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72410fdc0 con 0x7ff72410f040 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.046+0000 7ff729a00700 1 --2- 172.21.15.37:0/1858392172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff72410f460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff7180098a0 tx=0x7ff718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fa93526bd22171 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.046+0000 7ff723e00700 1 -- 172.21.15.37:0/1858392172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff718004160 con 0x7ff72410f040 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.046+0000 7ff723e00700 1 -- 172.21.15.37:0/1858392172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff718004300 con 0x7ff72410f040 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.046+0000 7ff723e00700 1 -- 172.21.15.37:0/1858392172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff718011af0 con 0x7ff72410f040 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 -- 172.21.15.37:0/1858392172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 msgr2=0x7ff72410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 --2- 172.21.15.37:0/1858392172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff72410f460 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff7180098a0 tx=0x7ff718009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 -- 172.21.15.37:0/1858392172 shutdown_connections 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 --2- 172.21.15.37:0/1858392172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff72410f460 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 -- 172.21.15.37:0/1858392172 >> 172.21.15.37:0/1858392172 conn(0x7ff72407ddf0 msgr2=0x7ff724080210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 -- 172.21.15.37:0/1858392172 shutdown_connections 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.047+0000 7ff72c400700 1 -- 172.21.15.37:0/1858392172 wait complete. 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.048+0000 7ff72c400700 1 Processor -- start 2024-04-26T01:45:23.384 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.049+0000 7ff72c400700 1 -- start start 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.049+0000 7ff72c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff7241a2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.049+0000 7ff72c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff724112ff0 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.049+0000 7ff729a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff7241a2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.049+0000 7ff729a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff7241a2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60418/0 (socket says 172.21.15.37:60418) 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.049+0000 7ff729a00700 1 -- 172.21.15.37:0/3386858874 learned_addr learned my addr 172.21.15.37:0/3386858874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff729a00700 1 -- 172.21.15.37:0/3386858874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7241a2fa0 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff729a00700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff7241a2a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff718003f40 tx=0x7ff718011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7180038b0 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ff718003a50 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7241a3230 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff71801a5c0 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.050+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7241a5f20 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.052+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7ff718024430 con 0x7ff72410f040 2024-04-26T01:45:23.385 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.052+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff724113170 con 0x7ff72410f040 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.052+0000 7ff722000700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff70c0395b0 0x7ff70c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.052+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff71804db40 con 0x7ff72410f040 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.053+0000 7ff729000700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff70c0395b0 0x7ff70c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.053+0000 7ff729000700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff70c0395b0 0x7ff70c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff710006fd0 tx=0x7ff710006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.058+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff71804d490 con 0x7ff72410f040 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:16.293+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7ff72407ffb0 con 0x7ff70c0395b0 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.604+0000 7ff722000700 1 -- 172.21.15.37:0/3386858874 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff72407ffb0 con 0x7ff70c0395b0 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff70c0395b0 msgr2=0x7ff70c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff70c0395b0 0x7ff70c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff710006fd0 tx=0x7ff710006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 msgr2=0x7ff7241a2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff7241a2a60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff718003f40 tx=0x7ff718011890 comp rx=0 tx=0).stop 2024-04-26T01:45:23.386 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 shutdown_connections 2024-04-26T01:45:23.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7ff70c0395b0 0x7ff70c03ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:23.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 --2- 172.21.15.37:0/3386858874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff72410f040 0x7ff7241a2a60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:23.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 >> 172.21.15.37:0/3386858874 conn(0x7ff72407ddf0 msgr2=0x7ff72407f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:23.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.609+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 shutdown_connections 2024-04-26T01:45:23.387 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:22.610+0000 7ff72c400700 1 -- 172.21.15.37:0/3386858874 wait complete. 2024-04-26T01:45:23.387 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-04-26T01:45:23.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:23 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:23.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:23 smithi037 ceph-mon[27906]: Added host smithi037 2024-04-26T01:45:23.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:23 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:25.367 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T01:45:25.370 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.062+0000 7f53d2400700 1 Processor -- start 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.062+0000 7f53d2400700 1 -- start start 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.063+0000 7f53d2400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc1089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.063+0000 7f53d2400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc108fb0 con 0x7f53cc1085c0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.063+0000 7f53cb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc1089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.063+0000 7f53cb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc1089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53416/0 (socket says 172.21.15.37:53416) 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.063+0000 7f53cb400700 1 -- 172.21.15.37:0/2033676799 learned_addr learned my addr 172.21.15.37:0/2033676799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.064+0000 7f53cb400700 1 -- 172.21.15.37:0/2033676799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc1097c0 con 0x7f53cc1085c0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.064+0000 7f53cb400700 1 --2- 172.21.15.37:0/2033676799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc1089e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0098a0 tx=0x7f53bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e36b2f3fbc631 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.064+0000 7f53caa00700 1 -- 172.21.15.37:0/2033676799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53bc004160 con 0x7f53cc1085c0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.064+0000 7f53caa00700 1 -- 172.21.15.37:0/2033676799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53bc004300 con 0x7f53cc1085c0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.064+0000 7f53caa00700 1 -- 172.21.15.37:0/2033676799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53bc011af0 con 0x7f53cc1085c0 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 -- 172.21.15.37:0/2033676799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 msgr2=0x7f53cc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:25.371 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 --2- 172.21.15.37:0/2033676799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc1089e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0098a0 tx=0x7f53bc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 -- 172.21.15.37:0/2033676799 shutdown_connections 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 --2- 172.21.15.37:0/2033676799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc1089e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 -- 172.21.15.37:0/2033676799 >> 172.21.15.37:0/2033676799 conn(0x7f53cc103be0 msgr2=0x7f53cc106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 -- 172.21.15.37:0/2033676799 shutdown_connections 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.065+0000 7f53d2400700 1 -- 172.21.15.37:0/2033676799 wait complete. 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.066+0000 7f53d2400700 1 Processor -- start 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.066+0000 7f53d2400700 1 -- start start 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.067+0000 7f53d2400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc07ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.067+0000 7f53d2400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53cc10c9f0 con 0x7f53cc1085c0 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.067+0000 7f53cb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc07ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.067+0000 7f53cb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc07ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53426/0 (socket says 172.21.15.37:53426) 2024-04-26T01:45:25.372 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.067+0000 7f53cb400700 1 -- 172.21.15.37:0/891907722 learned_addr learned my addr 172.21.15.37:0/891907722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:25.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53cb400700 1 -- 172.21.15.37:0/891907722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53cc07cf80 con 0x7f53cc1085c0 2024-04-26T01:45:25.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53cb400700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc07ca40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0117b0 tx=0x7f53bc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:25.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53bc003af0 con 0x7f53cc1085c0 2024-04-26T01:45:25.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f53bc020070 con 0x7f53cc1085c0 2024-04-26T01:45:25.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53cc07b170 con 0x7f53cc1085c0 2024-04-26T01:45:25.373 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f53bc019630 con 0x7f53cc1085c0 2024-04-26T01:45:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.068+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53cc07b6b0 con 0x7f53cc1085c0 2024-04-26T01:45:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.069+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f53bc01a9c0 con 0x7f53cc1085c0 2024-04-26T01:45:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.070+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53cc0640a0 con 0x7f53cc1085c0 2024-04-26T01:45:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.070+0000 7f53c8c00700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f53b0039600 0x7f53b003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.070+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f53bc04db30 con 0x7f53cc1085c0 2024-04-26T01:45:25.375 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.071+0000 7f53c2a00700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f53b0039600 0x7f53b003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.071+0000 7f53c2a00700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f53b0039600 0x7f53b003bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53b400ad30 tx=0x7f53b40093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.076+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53bc0042a0 con 0x7f53cc1085c0 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.338+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f53cc104e70 con 0x7f53b0039600 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.345+0000 7f53c8c00700 1 -- 172.21.15.37:0/891907722 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f53cc104e70 con 0x7f53b0039600 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f53b0039600 msgr2=0x7f53b003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f53b0039600 0x7f53b003bac0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53b400ad30 tx=0x7f53b40093f0 comp rx=0 tx=0).stop 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 msgr2=0x7f53cc07ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc07ca40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f53bc0117b0 tx=0x7f53bc011b70 comp rx=0 tx=0).stop 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 shutdown_connections 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f53b0039600 0x7f53b003bac0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 --2- 172.21.15.37:0/891907722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f53cc1085c0 0x7f53cc07ca40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.347+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 >> 172.21.15.37:0/891907722 conn(0x7f53cc103be0 msgr2=0x7f53cc104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.348+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 shutdown_connections 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:24.348+0000 7f53d2400700 1 -- 172.21.15.37:0/891907722 wait complete. 2024-04-26T01:45:25.376 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-04-26T01:45:25.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:25 smithi037 ceph-mon[27906]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:25.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:25 smithi037 ceph-mon[27906]: Saving service mon spec with placement count:5 2024-04-26T01:45:25.697 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:25 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:27.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:27 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:27.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:27 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:27.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:27 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.637+0000 7fdb4da00700 1 Processor -- start 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.637+0000 7fdb4da00700 1 -- start start 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.638+0000 7fdb4da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.638+0000 7fdb4da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4807c8f0 con 0x7fdb4807de70 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.638+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:27.699 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.638+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53428/0 (socket says 172.21.15.37:53428) 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.638+0000 7fdb46a00700 1 -- 172.21.15.37:0/3136456103 learned_addr learned my addr 172.21.15.37:0/3136456103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.639+0000 7fdb46a00700 1 -- 172.21.15.37:0/3136456103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4807ca70 con 0x7fdb4807de70 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.639+0000 7fdb46a00700 1 --2- 172.21.15.37:0/3136456103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4807c320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb380098a0 tx=0x7fdb38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a71d2d483cba61 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.639+0000 7fdb45600700 1 -- 172.21.15.37:0/3136456103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb38004160 con 0x7fdb4807de70 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.639+0000 7fdb45600700 1 -- 172.21.15.37:0/3136456103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb38004300 con 0x7fdb4807de70 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.639+0000 7fdb45600700 1 -- 172.21.15.37:0/3136456103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb38011a20 con 0x7fdb4807de70 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 -- 172.21.15.37:0/3136456103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 msgr2=0x7fdb4807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 --2- 172.21.15.37:0/3136456103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4807c320 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdb380098a0 tx=0x7fdb38009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 -- 172.21.15.37:0/3136456103 shutdown_connections 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 --2- 172.21.15.37:0/3136456103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4807c320 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 -- 172.21.15.37:0/3136456103 >> 172.21.15.37:0/3136456103 conn(0x7fdb48103c30 msgr2=0x7fdb48106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:27.700 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 -- 172.21.15.37:0/3136456103 shutdown_connections 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.640+0000 7fdb4da00700 1 -- 172.21.15.37:0/3136456103 wait complete. 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.641+0000 7fdb4da00700 1 Processor -- start 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.642+0000 7fdb4da00700 1 -- start start 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.642+0000 7fdb4da00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.642+0000 7fdb4da00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdb4810eb90 con 0x7fdb4807de70 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.642+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.642+0000 7fdb46a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53442/0 (socket says 172.21.15.37:53442) 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.642+0000 7fdb46a00700 1 -- 172.21.15.37:0/1507190139 learned_addr learned my addr 172.21.15.37:0/1507190139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb46a00700 1 -- 172.21.15.37:0/1507190139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4819eb70 con 0x7fdb4807de70 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb46a00700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4819e630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb38000c00 tx=0x7fdb38011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb38003aa0 con 0x7fdb4807de70 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb3801b8f0 con 0x7fdb4807de70 2024-04-26T01:45:27.701 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdb38023870 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4819ee00 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.643+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a1af0 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.644+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4810ed10 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.645+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdb3801b440 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.646+0000 7fdb3f400700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fdb2c039600 0x7fdb2c03bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.646+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdb3804df70 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.646+0000 7fdb46000700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fdb2c039600 0x7fdb2c03bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.647+0000 7fdb46000700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fdb2c039600 0x7fdb2c03bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006fd0 tx=0x7fdb34006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.652+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb38028060 con 0x7fdb4807de70 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.901+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdb48104e10 con 0x7fdb2c039600 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.908+0000 7fdb3f400700 1 -- 172.21.15.37:0/1507190139 <== mgr.14120 v2:172.21.15.37:6800/2267935573 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdb48104e10 con 0x7fdb2c039600 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fdb2c039600 msgr2=0x7fdb2c03bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fdb2c039600 0x7fdb2c03bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdb34006fd0 tx=0x7fdb34006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:27.702 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 msgr2=0x7fdb4819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4819e630 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdb38000c00 tx=0x7fdb38011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 shutdown_connections 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fdb2c039600 0x7fdb2c03bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 --2- 172.21.15.37:0/1507190139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdb4807de70 0x7fdb4819e630 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 >> 172.21.15.37:0/1507190139 conn(0x7fdb48103c30 msgr2=0x7fdb48104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 shutdown_connections 2024-04-26T01:45:27.703 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:26.911+0000 7fdb4da00700 1 -- 172.21.15.37:0/1507190139 wait complete. 2024-04-26T01:45:28.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:28 smithi037 ceph-mon[27906]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:45:28.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:28 smithi037 ceph-mon[27906]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:28.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:28 smithi037 ceph-mon[27906]: Saving service mgr spec with placement count:2 2024-04-26T01:45:29.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.270+0000 7faab0400700 1 Processor -- start 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.270+0000 7faab0400700 1 -- start start 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.271+0000 7faab0400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa8107270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.271+0000 7faab0400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8063d40 con 0x7faaa8106e50 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.271+0000 7faaada00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa8107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.271+0000 7faaada00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa8107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53450/0 (socket says 172.21.15.37:53450) 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.271+0000 7faaada00700 1 -- 172.21.15.37:0/1689190426 learned_addr learned my addr 172.21.15.37:0/1689190426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.272+0000 7faaada00700 1 -- 172.21.15.37:0/1689190426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa81077b0 con 0x7faaa8106e50 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.272+0000 7faaada00700 1 --2- 172.21.15.37:0/1689190426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa8107270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0098a0 tx=0x7faa9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed8e04bbd77d5827 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:29.774 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.273+0000 7faaa7e00700 1 -- 172.21.15.37:0/1689190426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa9c004160 con 0x7faaa8106e50 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.273+0000 7faaa7e00700 1 -- 172.21.15.37:0/1689190426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa9c004300 con 0x7faaa8106e50 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.273+0000 7faaa7e00700 1 -- 172.21.15.37:0/1689190426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa9c011a20 con 0x7faaa8106e50 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.274+0000 7faab0400700 1 -- 172.21.15.37:0/1689190426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 msgr2=0x7faaa8107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.274+0000 7faab0400700 1 --2- 172.21.15.37:0/1689190426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa8107270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7faa9c0098a0 tx=0x7faa9c009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.274+0000 7faab0400700 1 -- 172.21.15.37:0/1689190426 shutdown_connections 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.274+0000 7faab0400700 1 --2- 172.21.15.37:0/1689190426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa8107270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.274+0000 7faab0400700 1 -- 172.21.15.37:0/1689190426 >> 172.21.15.37:0/1689190426 conn(0x7faaa8102ab0 msgr2=0x7faaa8104ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.274+0000 7faab0400700 1 -- 172.21.15.37:0/1689190426 shutdown_connections 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.275+0000 7faab0400700 1 -- 172.21.15.37:0/1689190426 wait complete. 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.276+0000 7faab0400700 1 Processor -- start 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.276+0000 7faab0400700 1 -- start start 2024-04-26T01:45:29.777 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.276+0000 7faab0400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.276+0000 7faab0400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faaa8108820 con 0x7faaa8106e50 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.276+0000 7faaada00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.277+0000 7faaada00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53460/0 (socket says 172.21.15.37:53460) 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.277+0000 7faaada00700 1 -- 172.21.15.37:0/614282400 learned_addr learned my addr 172.21.15.37:0/614282400 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.277+0000 7faaada00700 1 -- 172.21.15.37:0/614282400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa819eb70 con 0x7faaa8106e50 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.277+0000 7faaada00700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa819e630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa9c000c00 tx=0x7faa9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.278+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa9c003aa0 con 0x7faaa8106e50 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.278+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa9c01b8f0 con 0x7faaa8106e50 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.278+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa9c023870 con 0x7faaa8106e50 2024-04-26T01:45:29.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.278+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa819ee00 con 0x7faaa8106e50 2024-04-26T01:45:29.780 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.278+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa81a1af0 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.279+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7faa9c01b440 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.279+0000 7faaa6000700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7faa90039600 0x7faa9003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.279+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa9c04de40 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.280+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa80640a0 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.280+0000 7faaad000700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7faa90039600 0x7faa9003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.283+0000 7faaad000700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7faa90039600 0x7faa9003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faa94006fd0 tx=0x7faa94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.286+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa9c01f030 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.530+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7faaa802e050 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.534+0000 7faaa6000700 1 -- 172.21.15.37:0/614282400 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7faa9c00be50 con 0x7faaa8106e50 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.537+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7faa90039600 msgr2=0x7faa9003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.537+0000 7faab0400700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7faa90039600 0x7faa9003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7faa94006fd0 tx=0x7faa94006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.537+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 msgr2=0x7faaa819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:29.781 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.537+0000 7faab0400700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa819e630 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa9c000c00 tx=0x7faa9c011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:29.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.538+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 shutdown_connections 2024-04-26T01:45:29.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.538+0000 7faab0400700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7faa90039600 0x7faa9003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:29.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.538+0000 7faab0400700 1 --2- 172.21.15.37:0/614282400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faaa8106e50 0x7faaa819e630 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:29.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.538+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 >> 172.21.15.37:0/614282400 conn(0x7faaa8102ab0 msgr2=0x7faaa8195270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:29.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.538+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 shutdown_connections 2024-04-26T01:45:29.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:28.538+0000 7faab0400700 1 -- 172.21.15.37:0/614282400 wait complete. 2024-04-26T01:45:29.939 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:29 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/614282400' entity='client.admin' 2024-04-26T01:45:30.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:30 smithi037 ceph-mon[27906]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:45:30.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:30 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:31.347 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.352+0000 7f47f0e00700 1 Processor -- start 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.353+0000 7f47f0e00700 1 -- start start 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.353+0000 7f47f0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec07b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.353+0000 7f47f0e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec07b780 con 0x7f47ec07cd50 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.353+0000 7f47e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec07b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.354+0000 7f47e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec07b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56754/0 (socket says 172.21.15.37:56754) 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.354+0000 7f47e9e00700 1 -- 172.21.15.37:0/3003893916 learned_addr learned my addr 172.21.15.37:0/3003893916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.354+0000 7f47e9e00700 1 -- 172.21.15.37:0/3003893916 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec07b900 con 0x7f47ec07cd50 2024-04-26T01:45:31.348 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.354+0000 7f47e9e00700 1 --2- 172.21.15.37:0/3003893916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec07b1b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0098a0 tx=0x7f47dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf1020571b63516 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.355+0000 7f47e8a00700 1 -- 172.21.15.37:0/3003893916 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc004160 con 0x7f47ec07cd50 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.355+0000 7f47e8a00700 1 -- 172.21.15.37:0/3003893916 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc004300 con 0x7f47ec07cd50 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.355+0000 7f47e8a00700 1 -- 172.21.15.37:0/3003893916 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc011a20 con 0x7f47ec07cd50 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 -- 172.21.15.37:0/3003893916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 msgr2=0x7f47ec07b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 --2- 172.21.15.37:0/3003893916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec07b1b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0098a0 tx=0x7f47dc009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 -- 172.21.15.37:0/3003893916 shutdown_connections 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 --2- 172.21.15.37:0/3003893916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec07b1b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 -- 172.21.15.37:0/3003893916 >> 172.21.15.37:0/3003893916 conn(0x7f47ec103bf0 msgr2=0x7f47ec106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 -- 172.21.15.37:0/3003893916 shutdown_connections 2024-04-26T01:45:31.349 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.356+0000 7f47f0e00700 1 -- 172.21.15.37:0/3003893916 wait complete. 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.357+0000 7f47f0e00700 1 Processor -- start 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.357+0000 7f47f0e00700 1 -- start start 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.357+0000 7f47f0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec19e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.357+0000 7f47f0e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47ec10ebf0 con 0x7f47ec07cd50 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.358+0000 7f47e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec19e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.358+0000 7f47e9e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec19e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56762/0 (socket says 172.21.15.37:56762) 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.358+0000 7f47e9e00700 1 -- 172.21.15.37:0/4242297603 learned_addr learned my addr 172.21.15.37:0/4242297603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.358+0000 7f47e9e00700 1 -- 172.21.15.37:0/4242297603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47ec19eaf0 con 0x7f47ec07cd50 2024-04-26T01:45:31.350 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.359+0000 7f47e9e00700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec19e5b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.359+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc003aa0 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.359+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47dc01b8f0 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.359+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47dc023870 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.359+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47ec19ed80 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.359+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47ec1a1a70 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.360+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f47dc01b440 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.361+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47ec0640a0 con 0x7f47ec07cd50 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.361+0000 7f47e2a00700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f47d0039600 0x7f47d003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:31.351 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.361+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f47dc04e300 con 0x7f47ec07cd50 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.361+0000 7f47e9400700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f47d0039600 0x7f47d003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.362+0000 7f47e9400700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f47d0039600 0x7f47d003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f47d8006fd0 tx=0x7f47d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.367+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47dc01f030 con 0x7f47ec07cd50 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.614+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f47ec1a1c40 con 0x7f47ec07cd50 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.617+0000 7f47e2a00700 1 -- 172.21.15.37:0/4242297603 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f47dc04dc50 con 0x7f47ec07cd50 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f47d0039600 msgr2=0x7f47d003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f47d0039600 0x7f47d003bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f47d8006fd0 tx=0x7f47d8006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 msgr2=0x7f47ec19e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec19e5b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f47dc000c00 tx=0x7f47dc011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:31.352 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 shutdown_connections 2024-04-26T01:45:31.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f47d0039600 0x7f47d003bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:31.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 --2- 172.21.15.37:0/4242297603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47ec07cd50 0x7f47ec19e5b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:31.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.620+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 >> 172.21.15.37:0/4242297603 conn(0x7f47ec103bf0 msgr2=0x7f47ec104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:31.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.621+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 shutdown_connections 2024-04-26T01:45:31.353 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:30.621+0000 7f47f0e00700 1 -- 172.21.15.37:0/4242297603 wait complete. 2024-04-26T01:45:31.353 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-04-26T01:45:31.920 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:31 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4242297603' entity='client.admin' 2024-04-26T01:45:32.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:32 smithi037 ceph-mon[27906]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:45:32.905 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:32 smithi037 ceph-mon[27906]: from='mgr.14120 172.21.15.37:0/909318911' entity='mgr.x' 2024-04-26T01:45:32.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:32 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1197376929' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T01:45:33.426 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: ignoring --setuser ceph since I am not root 2024-04-26T01:45:33.426 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: ignoring --setgroup ceph since I am not root 2024-04-26T01:45:33.426 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:33.343+0000 7f64c558c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:45:33.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:33.605+0000 7f64c558c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:45:34.024 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:33.740+0000 7f64c558c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:45:34.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:34 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1197376929' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T01:45:34.317 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:34 smithi037 ceph-mon[27906]: mgrmap e9: x(active, since 26s) 2024-04-26T01:45:34.317 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:34.156+0000 7f64c558c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:45:34.317 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:34.272+0000 7f64c558c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.003+0000 7fa0aa400700 1 Processor -- start 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.003+0000 7fa0aa400700 1 -- start start 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.004+0000 7fa0aa400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a4108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.004+0000 7fa0aa400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4108fd0 con 0x7fa0a41085e0 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.004+0000 7fa0a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a4108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.004+0000 7fa0a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a4108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56764/0 (socket says 172.21.15.37:56764) 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.004+0000 7fa0a3400700 1 -- 172.21.15.37:0/1796977135 learned_addr learned my addr 172.21.15.37:0/1796977135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:34.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.005+0000 7fa0a3400700 1 -- 172.21.15.37:0/1796977135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a4109790 con 0x7fa0a41085e0 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.005+0000 7fa0a3400700 1 --2- 172.21.15.37:0/1796977135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a4108a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098a0 tx=0x7fa094009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c749aa6fcd85afa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.006+0000 7fa0a2000700 1 -- 172.21.15.37:0/1796977135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa094004160 con 0x7fa0a41085e0 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.006+0000 7fa0a2000700 1 -- 172.21.15.37:0/1796977135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa094004300 con 0x7fa0a41085e0 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.006+0000 7fa0a2000700 1 -- 172.21.15.37:0/1796977135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa094011a20 con 0x7fa0a41085e0 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 -- 172.21.15.37:0/1796977135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 msgr2=0x7fa0a4108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 --2- 172.21.15.37:0/1796977135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a4108a00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa0940098a0 tx=0x7fa094009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 -- 172.21.15.37:0/1796977135 shutdown_connections 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 --2- 172.21.15.37:0/1796977135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a4108a00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 -- 172.21.15.37:0/1796977135 >> 172.21.15.37:0/1796977135 conn(0x7fa0a4103c60 msgr2=0x7fa0a4106080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 -- 172.21.15.37:0/1796977135 shutdown_connections 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.007+0000 7fa0aa400700 1 -- 172.21.15.37:0/1796977135 wait complete. 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.008+0000 7fa0aa400700 1 Processor -- start 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.008+0000 7fa0aa400700 1 -- start start 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.009+0000 7fa0aa400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a407db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:34.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.009+0000 7fa0aa400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a410cb40 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.009+0000 7fa0a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a407db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.009+0000 7fa0a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a407db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56772/0 (socket says 172.21.15.37:56772) 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.009+0000 7fa0a3400700 1 -- 172.21.15.37:0/1197376929 learned_addr learned my addr 172.21.15.37:0/1197376929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa0a3400700 1 -- 172.21.15.37:0/1197376929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a407e090 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa0a3400700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a407db50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0940119f0 tx=0x7fa094011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0940039e0 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09401b8f0 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a407c280 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0940237a0 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.010+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a407c7c0 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.011+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0a410ccc0 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.012+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa09401b440 con 0x7fa0a41085e0 2024-04-26T01:45:34.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.012+0000 7fa09be00700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fa0880395b0 0x7fa08803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.012+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa09404dec0 con 0x7fa0a41085e0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.012+0000 7fa0a2a00700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fa0880395b0 0x7fa08803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.013+0000 7fa0a2a00700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fa0880395b0 0x7fa08803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa090006fd0 tx=0x7fa090006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.018+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa09401f030 con 0x7fa0a41085e0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:32.319+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fa0a407ce60 con 0x7fa0a41085e0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.024+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fa094018350 con 0x7fa0a41085e0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.028+0000 7fa09be00700 1 -- 172.21.15.37:0/1197376929 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fa094016ca0 con 0x7fa0a41085e0 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fa0880395b0 msgr2=0x7fa08803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fa0880395b0 0x7fa08803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa090006fd0 tx=0x7fa090006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 msgr2=0x7fa0a407db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a407db50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa0940119f0 tx=0x7fa094011ad0 comp rx=0 tx=0).stop 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 shutdown_connections 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7fa0880395b0 0x7fa08803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:34.663 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 --2- 172.21.15.37:0/1197376929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0a41085e0 0x7fa0a407db50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:34.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.030+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 >> 172.21.15.37:0/1197376929 conn(0x7fa0a4103c60 msgr2=0x7fa0a4104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:34.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.031+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 shutdown_connections 2024-04-26T01:45:34.664 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:33.031+0000 7fa0aa400700 1 -- 172.21.15.37:0/1197376929 wait complete. 2024-04-26T01:45:35.935 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:35 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3272952745' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T01:45:36.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:35 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:35.937+0000 7f64c558c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:45:36.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:36.172+0000 7f64c558c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.292+0000 7f5d99600700 1 Processor -- start 2024-04-26T01:45:36.341 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.292+0000 7f5d99600700 1 -- start start 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.293+0000 7f5d99600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.293+0000 7f5d99600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9407b7c0 con 0x7f5d9407cd90 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.293+0000 7f5d92a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.293+0000 7f5d92a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9407b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56790/0 (socket says 172.21.15.37:56790) 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.293+0000 7f5d92a00700 1 -- 172.21.15.37:0/3823588426 learned_addr learned my addr 172.21.15.37:0/3823588426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.294+0000 7f5d92a00700 1 -- 172.21.15.37:0/3823588426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9407b940 con 0x7f5d9407cd90 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.294+0000 7f5d92a00700 1 --2- 172.21.15.37:0/3823588426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9407b1f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d840098a0 tx=0x7f5d84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecce147be81580f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.294+0000 7f5d91600700 1 -- 172.21.15.37:0/3823588426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d84004160 con 0x7f5d9407cd90 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.294+0000 7f5d91600700 1 -- 172.21.15.37:0/3823588426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d84004300 con 0x7f5d9407cd90 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.295+0000 7f5d91600700 1 -- 172.21.15.37:0/3823588426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d84011a20 con 0x7f5d9407cd90 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.295+0000 7f5d99600700 1 -- 172.21.15.37:0/3823588426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 msgr2=0x7f5d9407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.295+0000 7f5d99600700 1 --2- 172.21.15.37:0/3823588426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9407b1f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d840098a0 tx=0x7f5d84009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:36.342 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.296+0000 7f5d99600700 1 -- 172.21.15.37:0/3823588426 shutdown_connections 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.296+0000 7f5d99600700 1 --2- 172.21.15.37:0/3823588426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9407b1f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.296+0000 7f5d99600700 1 -- 172.21.15.37:0/3823588426 >> 172.21.15.37:0/3823588426 conn(0x7f5d94103bf0 msgr2=0x7f5d94106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.296+0000 7f5d99600700 1 -- 172.21.15.37:0/3823588426 shutdown_connections 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.296+0000 7f5d99600700 1 -- 172.21.15.37:0/3823588426 wait complete. 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d99600700 1 Processor -- start 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d99600700 1 -- start start 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d99600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d99600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9410ebb0 con 0x7f5d9407cd90 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d92a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d92a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56806/0 (socket says 172.21.15.37:56806) 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.297+0000 7f5d92a00700 1 -- 172.21.15.37:0/3272952745 learned_addr learned my addr 172.21.15.37:0/3272952745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d92a00700 1 -- 172.21.15.37:0/3272952745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9419eb40 con 0x7f5d9407cd90 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d92a00700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9419e600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d840116e0 tx=0x7f5d84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:36.343 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d840039e0 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d8401b8f0 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d84023760 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9419edd0 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.298+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d941a1ac0 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.300+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5d8401b440 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.300+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d940640a0 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.300+0000 7f5d8b400700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 0x7f5d7803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.300+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5d8404e040 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.300+0000 7f5d92000700 1 -- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 msgr2=0x7f5d7803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.300+0000 7f5d92000700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 0x7f5d7803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.306+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5d8401f030 con 0x7f5d9407cd90 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.501+0000 7f5d92000700 1 -- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 msgr2=0x7f5d7803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.501+0000 7f5d92000700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 0x7f5d7803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:45:36.344 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.622+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f5d941a1c90 con 0x7f5d9407cd90 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.623+0000 7f5d8b400700 1 -- 172.21.15.37:0/3272952745 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f5d840292a0 con 0x7f5d9407cd90 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 msgr2=0x7f5d7803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 0x7f5d7803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 msgr2=0x7f5d9419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9419e600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5d840116e0 tx=0x7f5d84011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 shutdown_connections 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f5d780395b0 0x7f5d7803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 --2- 172.21.15.37:0/3272952745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5d9407cd90 0x7f5d9419e600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 >> 172.21.15.37:0/3272952745 conn(0x7f5d94103bf0 msgr2=0x7f5d94104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 shutdown_connections 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:35.628+0000 7f5d99600700 1 -- 172.21.15.37:0/3272952745 wait complete. 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-04-26T01:45:36.345 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-04-26T01:45:36.646 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:36.400+0000 7f64c558c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:45:36.921 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:36.808+0000 7f64c558c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:45:37.198 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:36.938+0000 7f64c558c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:45:37.198 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:37.199+0000 7f64c558c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:45:37.956 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:37.462+0000 7f64c558c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:45:38.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:38.273+0000 7f64c558c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:45:39.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:38.845+0000 7f64c558c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:45:39.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:38.977+0000 7f64c558c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:45:39.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:39.253+0000 7f64c558c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:45:40.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:39.925+0000 7f64c558c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:45:40.207 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:40.054+0000 7f64c558c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:45:40.572 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:40.398+0000 7f64c558c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:45:40.881 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:40.589+0000 7f64c558c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:45:40.881 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:40.722+0000 7f64c558c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:45:40.881 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:40.882+0000 7f64c558c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:45:41.707 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:41.265+0000 7f64c558c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:45:42.357 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:42.062+0000 7f64c558c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:45:42.706 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:45:42.358+0000 7f64c558c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: Active manager daemon x restarted 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: Activating manager daemon x 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: mgrmap e10: x(active, starting, since 0.0048542s) 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:45:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:45:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:45:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:45:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:42 smithi037 ceph-mon[27906]: Manager daemon x is now available 2024-04-26T01:45:43.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:45:43.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:45:43.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:45:43.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:42] ENGINE Bus STARTING 2024-04-26T01:45:43.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:42] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T01:45:43.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:42] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T01:45:43.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: [26/Apr/2024:01:45:42] ENGINE Bus STARTED 2024-04-26T01:45:43.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:43 smithi037 ceph-mon[27906]: mgrmap e11: x(active, since 1.01487s) 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.939+0000 7f31cc000700 1 Processor -- start 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.940+0000 7f31cc000700 1 -- start start 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.940+0000 7f31cc000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.940+0000 7f31cc000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31c407b7c0 con 0x7f31c407cd90 2024-04-26T01:45:44.092 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.940+0000 7f31c9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.941+0000 7f31c9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c407b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56814/0 (socket says 172.21.15.37:56814) 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.941+0000 7f31c9600700 1 -- 172.21.15.37:0/2940890098 learned_addr learned my addr 172.21.15.37:0/2940890098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.941+0000 7f31c9600700 1 -- 172.21.15.37:0/2940890098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c407b940 con 0x7f31c407cd90 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.941+0000 7f31c9600700 1 --2- 172.21.15.37:0/2940890098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c407b1f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31b80098a0 tx=0x7f31b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6140d0cf4797f4d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.942+0000 7f31c3e00700 1 -- 172.21.15.37:0/2940890098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31b8004160 con 0x7f31c407cd90 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.942+0000 7f31c3e00700 1 -- 172.21.15.37:0/2940890098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b8004300 con 0x7f31c407cd90 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.942+0000 7f31c3e00700 1 -- 172.21.15.37:0/2940890098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31b8011a20 con 0x7f31c407cd90 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 -- 172.21.15.37:0/2940890098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 msgr2=0x7f31c407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 --2- 172.21.15.37:0/2940890098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c407b1f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f31b80098a0 tx=0x7f31b8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 -- 172.21.15.37:0/2940890098 shutdown_connections 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 --2- 172.21.15.37:0/2940890098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c407b1f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.093 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 -- 172.21.15.37:0/2940890098 >> 172.21.15.37:0/2940890098 conn(0x7f31c4103be0 msgr2=0x7f31c4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 -- 172.21.15.37:0/2940890098 shutdown_connections 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.943+0000 7f31cc000700 1 -- 172.21.15.37:0/2940890098 wait complete. 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.944+0000 7f31cc000700 1 Processor -- start 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.944+0000 7f31cc000700 1 -- start start 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.944+0000 7f31cc000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.944+0000 7f31cc000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31c410eba0 con 0x7f31c407cd90 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.945+0000 7f31c9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.945+0000 7f31c9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56824/0 (socket says 172.21.15.37:56824) 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.945+0000 7f31c9600700 1 -- 172.21.15.37:0/1659268013 learned_addr learned my addr 172.21.15.37:0/1659268013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.945+0000 7f31c9600700 1 -- 172.21.15.37:0/1659268013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c419ebc0 con 0x7f31c407cd90 2024-04-26T01:45:44.094 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.945+0000 7f31c9600700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c419e680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31b8000c00 tx=0x7f31b8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.946+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31b8003aa0 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.946+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31b801b8f0 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.946+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f31b8023890 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.946+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c419ee50 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.946+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c41a1b40 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.947+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f31b801b440 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.947+0000 7f31c2000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.947+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 --> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f31ac03c230 con 0x7f31ac039660 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.948+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f31b804df10 con 0x7f31c407cd90 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.948+0000 7f31c8c00700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 msgr2=0x7f31ac03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:36.948+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:37.149+0000 7f31c8c00700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 msgr2=0x7f31ac03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:44.095 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:37.149+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:37.550+0000 7f31c8c00700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 msgr2=0x7f31ac03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:37.550+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:38.351+0000 7f31c8c00700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 msgr2=0x7f31ac03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:38.351+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:39.953+0000 7f31c8c00700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 msgr2=0x7f31ac03bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2267935573 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:39.953+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:42.367+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f31b802c770 con 0x7f31c407cd90 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:42.367+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 msgr2=0x7f31ac03bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:42.367+0000 7f31c2000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.371+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f31b8028cf0 con 0x7f31c407cd90 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.372+0000 7f31c2000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31ac03cff0 0x7f31ac03f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.372+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f31b802ae50 con 0x7f31ac03cff0 2024-04-26T01:45:44.096 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.373+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31ac03cff0 0x7f31ac03f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.376+0000 7f31c8c00700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31ac03cff0 0x7f31ac03f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31b0003a10 tx=0x7f31b0009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.378+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f31b802ae50 con 0x7f31ac03cff0 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.386+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f31c4069b80 con 0x7f31ac03cff0 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.387+0000 7f31c2000700 1 -- 172.21.15.37:0/1659268013 <== mgr.14152 v2:172.21.15.37:6800/691602839 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f31c4069b80 con 0x7f31ac03cff0 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31ac03cff0 msgr2=0x7f31ac03f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31ac03cff0 0x7f31ac03f3e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f31b0003a10 tx=0x7f31b0009270 comp rx=0 tx=0).stop 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 msgr2=0x7f31c419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c419e680 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f31b8000c00 tx=0x7f31b8011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 shutdown_connections 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31ac03cff0 0x7f31ac03f3e0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31c407cd90 0x7f31c419e680 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 --2- 172.21.15.37:0/1659268013 >> [v2:172.21.15.37:6800/2267935573,v1:172.21.15.37:6801/2267935573] conn(0x7f31ac039660 0x7f31ac03bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 >> 172.21.15.37:0/1659268013 conn(0x7f31c4103be0 msgr2=0x7f31c4104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:44.097 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 shutdown_connections 2024-04-26T01:45:44.098 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:43.388+0000 7f31cc000700 1 -- 172.21.15.37:0/1659268013 wait complete. 2024-04-26T01:45:44.098 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-04-26T01:45:44.098 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-04-26T01:45:44.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:44 smithi037 ceph-mon[27906]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T01:45:44.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:44 smithi037 ceph-mon[27906]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T01:45:45.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:45 smithi037 ceph-mon[27906]: mgrmap e12: x(active, since 2s) 2024-04-26T01:45:45.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:45 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:45:45.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:45 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:45:46.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T01:45:46.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.917+0000 7fabed000700 1 Processor -- start 2024-04-26T01:45:46.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.918+0000 7fabed000700 1 -- start start 2024-04-26T01:45:46.062 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.918+0000 7fabed000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe81089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:46.063 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.918+0000 7fabed000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fabe8108fb0 con 0x7fabe81085c0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.919+0000 7fabe5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe81089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.919+0000 7fabe5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe81089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45896/0 (socket says 172.21.15.37:45896) 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.919+0000 7fabe5e00700 1 -- 172.21.15.37:0/1642419671 learned_addr learned my addr 172.21.15.37:0/1642419671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.919+0000 7fabe5e00700 1 -- 172.21.15.37:0/1642419671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe81097c0 con 0x7fabe81085c0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.920+0000 7fabe5e00700 1 --2- 172.21.15.37:0/1642419671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe81089e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabd80098a0 tx=0x7fabd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccfcfdb936fa653b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.920+0000 7fabe5400700 1 -- 172.21.15.37:0/1642419671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd8004160 con 0x7fabe81085c0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.920+0000 7fabe5400700 1 -- 172.21.15.37:0/1642419671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd8004300 con 0x7fabe81085c0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.920+0000 7fabe5400700 1 -- 172.21.15.37:0/1642419671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd8011a20 con 0x7fabe81085c0 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.920+0000 7fabed000700 1 -- 172.21.15.37:0/1642419671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 msgr2=0x7fabe81089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.920+0000 7fabed000700 1 --2- 172.21.15.37:0/1642419671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe81089e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fabd80098a0 tx=0x7fabd8009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:46.068 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.921+0000 7fabed000700 1 -- 172.21.15.37:0/1642419671 shutdown_connections 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.921+0000 7fabed000700 1 --2- 172.21.15.37:0/1642419671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe81089e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.921+0000 7fabed000700 1 -- 172.21.15.37:0/1642419671 >> 172.21.15.37:0/1642419671 conn(0x7fabe8103be0 msgr2=0x7fabe8106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.921+0000 7fabed000700 1 -- 172.21.15.37:0/1642419671 shutdown_connections 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.921+0000 7fabed000700 1 -- 172.21.15.37:0/1642419671 wait complete. 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.922+0000 7fabed000700 1 Processor -- start 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.922+0000 7fabed000700 1 -- start start 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.923+0000 7fabed000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe819c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.923+0000 7fabed000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fabe810cb70 con 0x7fabe81085c0 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.923+0000 7fabe5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe819c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.923+0000 7fabe5e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe819c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45912/0 (socket says 172.21.15.37:45912) 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.923+0000 7fabe5e00700 1 -- 172.21.15.37:0/229653976 learned_addr learned my addr 172.21.15.37:0/229653976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabe5e00700 1 -- 172.21.15.37:0/229653976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabe819ca00 con 0x7fabe81085c0 2024-04-26T01:45:46.069 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabe5e00700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe819c4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fabd8000c00 tx=0x7fabd8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd8003aa0 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabd801b8f0 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabd8023890 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabe819cc90 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.924+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabe819f980 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.926+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fabd801b440 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.926+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabe810ccf0 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.926+0000 7fabdf400700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fabc8039660 0x7fabc803bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.926+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabd804dff0 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.926+0000 7fabdd400700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fabc8039660 0x7fabc803bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.927+0000 7fabdd400700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fabc8039660 0x7fabc803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fabd0006fd0 tx=0x7fabd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:44.932+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabd801f030 con 0x7fabe81085c0 2024-04-26T01:45:46.070 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.173+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fabe8104e30 con 0x7fabc8039660 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.301+0000 7fabdf400700 1 -- 172.21.15.37:0/229653976 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fabe8104e30 con 0x7fabc8039660 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fabc8039660 msgr2=0x7fabc803bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fabc8039660 0x7fabc803bb20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fabd0006fd0 tx=0x7fabd0006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 msgr2=0x7fabe819c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe819c4c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fabd8000c00 tx=0x7fabd8011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 shutdown_connections 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fabc8039660 0x7fabc803bb20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 --2- 172.21.15.37:0/229653976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fabe81085c0 0x7fabe819c4c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 >> 172.21.15.37:0/229653976 conn(0x7fabe8103be0 msgr2=0x7fabe8104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 shutdown_connections 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:45.304+0000 7fabed000700 1 -- 172.21.15.37:0/229653976 wait complete. 2024-04-26T01:45:46.071 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-04-26T01:45:46.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:46 smithi037 ceph-mon[27906]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:46.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:45:47.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:47 smithi037 ceph-mon[27906]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:45:47.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:47 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:45:47.954 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$6hLKpC245OZmJI0.Wp7xRu.5plq6RlndVtpA.7Sy25jhWLa0wYGum", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714095947, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T01:45:47.954 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.689+0000 7fc55a200700 1 Processor -- start 2024-04-26T01:45:47.954 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.689+0000 7fc55a200700 1 -- start start 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.690+0000 7fc55a200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55410b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.690+0000 7fc55a200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc554063d60 con 0x7fc554108ea0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.690+0000 7fc558e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55410b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.690+0000 7fc558e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55410b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45928/0 (socket says 172.21.15.37:45928) 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.690+0000 7fc558e00700 1 -- 172.21.15.37:0/2666910891 learned_addr learned my addr 172.21.15.37:0/2666910891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.691+0000 7fc558e00700 1 -- 172.21.15.37:0/2666910891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55410b7d0 con 0x7fc554108ea0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.691+0000 7fc558e00700 1 --2- 172.21.15.37:0/2666910891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55410b290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5480098a0 tx=0x7fc548009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c32d8b22bc400 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.691+0000 7fc553400700 1 -- 172.21.15.37:0/2666910891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc548004160 con 0x7fc554108ea0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.691+0000 7fc553400700 1 -- 172.21.15.37:0/2666910891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc548004300 con 0x7fc554108ea0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.691+0000 7fc553400700 1 -- 172.21.15.37:0/2666910891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc548011a20 con 0x7fc554108ea0 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 -- 172.21.15.37:0/2666910891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 msgr2=0x7fc55410b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:47.955 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 --2- 172.21.15.37:0/2666910891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55410b290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc5480098a0 tx=0x7fc548009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 -- 172.21.15.37:0/2666910891 shutdown_connections 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 --2- 172.21.15.37:0/2666910891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55410b290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 -- 172.21.15.37:0/2666910891 >> 172.21.15.37:0/2666910891 conn(0x7fc5541028d0 msgr2=0x7fc554104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 -- 172.21.15.37:0/2666910891 shutdown_connections 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.692+0000 7fc55a200700 1 -- 172.21.15.37:0/2666910891 wait complete. 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.693+0000 7fc55a200700 1 Processor -- start 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.694+0000 7fc55a200700 1 -- start start 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.694+0000 7fc55a200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55419e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.694+0000 7fc55a200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc55410c7e0 con 0x7fc554108ea0 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.694+0000 7fc558e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55419e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.694+0000 7fc558e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55419e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45942/0 (socket says 172.21.15.37:45942) 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.694+0000 7fc558e00700 1 -- 172.21.15.37:0/4230141179 learned_addr learned my addr 172.21.15.37:0/4230141179 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.695+0000 7fc558e00700 1 -- 172.21.15.37:0/4230141179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc55419ea00 con 0x7fc554108ea0 2024-04-26T01:45:47.956 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.695+0000 7fc558e00700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55419e4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc548000c00 tx=0x7fc548011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.695+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc548003aa0 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.695+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc54801b8f0 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.695+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55419ec90 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.696+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc548023870 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.696+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5541a1980 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.697+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc554069b80 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.697+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fc54801b440 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.697+0000 7fc551600700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc53c039660 0x7fc53c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.698+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc54804dfe0 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.698+0000 7fc553e00700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc53c039660 0x7fc53c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.699+0000 7fc553e00700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc53c039660 0x7fc53c03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.703+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc54801f030 con 0x7fc554108ea0 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:46.963+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fc5541a1c10 con 0x7fc53c039660 2024-04-26T01:45:47.957 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.225+0000 7fc551600700 1 -- 172.21.15.37:0/4230141179 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fc5541a1c10 con 0x7fc53c039660 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc53c039660 msgr2=0x7fc53c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc53c039660 0x7fc53c03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc540006fd0 tx=0x7fc540006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 msgr2=0x7fc55419e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55419e4c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc548000c00 tx=0x7fc548011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 shutdown_connections 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc53c039660 0x7fc53c03bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 --2- 172.21.15.37:0/4230141179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc554108ea0 0x7fc55419e4c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 >> 172.21.15.37:0/4230141179 conn(0x7fc5541028d0 msgr2=0x7fc5541032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.228+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 shutdown_connections 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:47.229+0000 7fc55a200700 1 -- 172.21.15.37:0/4230141179 wait complete. 2024-04-26T01:45:47.958 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-04-26T01:45:48.877 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:45:50.035 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:49 smithi037 ceph-mon[27906]: mgrmap e13: x(active, since 6s) 2024-04-26T01:45:50.035 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:49 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1622592387' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T01:45:50.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T01:45:50.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.644+0000 7fd09de00700 1 Processor -- start 2024-04-26T01:45:50.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.644+0000 7fd09de00700 1 -- start start 2024-04-26T01:45:50.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.644+0000 7fd09de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09807b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:50.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.644+0000 7fd09de00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd09807b7a0 con 0x7fd09807cd70 2024-04-26T01:45:50.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.645+0000 7fd096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09807b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:50.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.645+0000 7fd096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09807b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45956/0 (socket says 172.21.15.37:45956) 2024-04-26T01:45:50.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.645+0000 7fd096a00700 1 -- 172.21.15.37:0/2093037547 learned_addr learned my addr 172.21.15.37:0/2093037547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.645+0000 7fd096a00700 1 -- 172.21.15.37:0/2093037547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09807b920 con 0x7fd09807cd70 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.646+0000 7fd096a00700 1 --2- 172.21.15.37:0/2093037547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09807b1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd0880098a0 tx=0x7fd088009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fb1970569a57b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.646+0000 7fd095600700 1 -- 172.21.15.37:0/2093037547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd088004160 con 0x7fd09807cd70 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.646+0000 7fd095600700 1 -- 172.21.15.37:0/2093037547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd088004300 con 0x7fd09807cd70 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.646+0000 7fd095600700 1 -- 172.21.15.37:0/2093037547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd088011a20 con 0x7fd09807cd70 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 -- 172.21.15.37:0/2093037547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 msgr2=0x7fd09807b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 --2- 172.21.15.37:0/2093037547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09807b1d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd0880098a0 tx=0x7fd088009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 -- 172.21.15.37:0/2093037547 shutdown_connections 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 --2- 172.21.15.37:0/2093037547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09807b1d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 -- 172.21.15.37:0/2093037547 >> 172.21.15.37:0/2093037547 conn(0x7fd098103be0 msgr2=0x7fd098106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 -- 172.21.15.37:0/2093037547 shutdown_connections 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.647+0000 7fd09de00700 1 -- 172.21.15.37:0/2093037547 wait complete. 2024-04-26T01:45:50.787 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.648+0000 7fd09de00700 1 Processor -- start 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.648+0000 7fd09de00700 1 -- start start 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.649+0000 7fd09de00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.649+0000 7fd09de00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd09810ebc0 con 0x7fd09807cd70 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.649+0000 7fd096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.649+0000 7fd096a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45972/0 (socket says 172.21.15.37:45972) 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.649+0000 7fd096a00700 1 -- 172.21.15.37:0/1622592387 learned_addr learned my addr 172.21.15.37:0/1622592387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.649+0000 7fd096a00700 1 -- 172.21.15.37:0/1622592387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09819ebe0 con 0x7fd09807cd70 2024-04-26T01:45:50.789 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.650+0000 7fd096a00700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09819e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd088000c00 tx=0x7fd088011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:50.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.650+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd088003aa0 con 0x7fd09807cd70 2024-04-26T01:45:50.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.650+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd08801b8f0 con 0x7fd09807cd70 2024-04-26T01:45:50.790 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.650+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd088023890 con 0x7fd09807cd70 2024-04-26T01:45:50.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.650+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09819ee70 con 0x7fd09807cd70 2024-04-26T01:45:50.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.650+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0981a1b60 con 0x7fd09807cd70 2024-04-26T01:45:50.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.652+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0980640a0 con 0x7fd09807cd70 2024-04-26T01:45:50.792 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.652+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd08801b440 con 0x7fd09807cd70 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.652+0000 7fd09ca00700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd07c039660 0x7fd07c03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.652+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd08804dff0 con 0x7fd09807cd70 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.653+0000 7fd096000700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd07c039660 0x7fd07c03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.653+0000 7fd096000700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd07c039660 0x7fd07c03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd084006fd0 tx=0x7fd084006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.658+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd08801f030 con 0x7fd09807cd70 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.783+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd08802c770 con 0x7fd09807cd70 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.894+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd0981a1d30 con 0x7fd09807cd70 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.895+0000 7fd09ca00700 1 -- 172.21.15.37:0/1622592387 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd08802a300 con 0x7fd09807cd70 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.898+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd07c039660 msgr2=0x7fd07c03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.898+0000 7fd09de00700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd07c039660 0x7fd07c03bb20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd084006fd0 tx=0x7fd084006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.898+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 msgr2=0x7fd09819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.898+0000 7fd09de00700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09819e6a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd088000c00 tx=0x7fd088011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.899+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 shutdown_connections 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.899+0000 7fd09de00700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd07c039660 0x7fd07c03bb20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:50.793 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.899+0000 7fd09de00700 1 --2- 172.21.15.37:0/1622592387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09807cd70 0x7fd09819e6a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:50.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.899+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 >> 172.21.15.37:0/1622592387 conn(0x7fd098103be0 msgr2=0x7fd098104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:50.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.899+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 shutdown_connections 2024-04-26T01:45:50.794 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:48.899+0000 7fd09de00700 1 -- 172.21.15.37:0/1622592387 wait complete. 2024-04-26T01:45:50.824 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T01:45:50.825 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-04-26T01:45:50.847 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T01:45:50.847 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-04-26T01:45:50.847 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-04-26T01:45:50.847 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037.front.sepia.ceph.com:8443/ 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout: Password: 1ebzugtjjg 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:50.851 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config directory 2024-04-26T01:45:50.853 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling autotune for osd_memory_target 2024-04-26T01:45:52.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.506+0000 7f5de2200700 1 Processor -- start 2024-04-26T01:45:52.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.506+0000 7f5de2200700 1 -- start start 2024-04-26T01:45:52.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.507+0000 7f5de2200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:52.492 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.507+0000 7f5de2200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc1109a0 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.508+0000 7f5de0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:52.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.508+0000 7f5de0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1103d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38962/0 (socket says 172.21.15.37:38962) 2024-04-26T01:45:52.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.508+0000 7f5de0e00700 1 -- 172.21.15.37:0/547723859 learned_addr learned my addr 172.21.15.37:0/547723859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:52.494 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.509+0000 7f5de0e00700 1 -- 172.21.15.37:0/547723859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc10ecb0 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.509+0000 7f5de0e00700 1 --2- 172.21.15.37:0/547723859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1103d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009e80 tx=0x7f5dd000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ec52221a58d398 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.509+0000 7f5ddb400700 1 -- 172.21.15.37:0/547723859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd0017070 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.509+0000 7f5ddb400700 1 -- 172.21.15.37:0/547723859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd000b8c0 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.509+0000 7f5ddb400700 1 -- 172.21.15.37:0/547723859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd0004490 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.510+0000 7f5de2200700 1 -- 172.21.15.37:0/547723859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 msgr2=0x7f5ddc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.510+0000 7f5de2200700 1 --2- 172.21.15.37:0/547723859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1103d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5dd0009e80 tx=0x7f5dd000b330 comp rx=0 tx=0).stop 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.510+0000 7f5de2200700 1 -- 172.21.15.37:0/547723859 shutdown_connections 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.510+0000 7f5de2200700 1 --2- 172.21.15.37:0/547723859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1103d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.510+0000 7f5de2200700 1 -- 172.21.15.37:0/547723859 >> 172.21.15.37:0/547723859 conn(0x7f5ddc07dbf0 msgr2=0x7f5ddc080010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.510+0000 7f5de2200700 1 -- 172.21.15.37:0/547723859 shutdown_connections 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.511+0000 7f5de2200700 1 -- 172.21.15.37:0/547723859 wait complete. 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.512+0000 7f5de2200700 1 Processor -- start 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.512+0000 7f5de2200700 1 -- start start 2024-04-26T01:45:52.495 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.512+0000 7f5de2200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1a28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.512+0000 7f5de2200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ddc112ef0 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.512+0000 7f5de0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1a28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.513+0000 7f5de0e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1a28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38970/0 (socket says 172.21.15.37:38970) 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.513+0000 7f5de0e00700 1 -- 172.21.15.37:0/3049362400 learned_addr learned my addr 172.21.15.37:0/3049362400 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.513+0000 7f5de0e00700 1 -- 172.21.15.37:0/3049362400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ddc1a2df0 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.513+0000 7f5de0e00700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1a28b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000b7b0 tx=0x7f5dd000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.514+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd0020020 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.514+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd0003b50 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.514+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ddc1a3080 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.514+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dd0017040 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.499 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.514+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ddc1a5d70 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.515+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ddc069b80 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.515+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5dd0023020 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.516+0000 7f5dd9600700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5dc40396f0 0x7f5dc403bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.516+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5dd000f050 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.516+0000 7f5ddbe00700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5dc40396f0 0x7f5dc403bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.517+0000 7f5ddbe00700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5dc40396f0 0x7f5dc403bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.523+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dd002a020 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.750+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f5ddc1a5f40 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.752+0000 7f5dd9600700 1 -- 172.21.15.37:0/3049362400 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f5dd002a200 con 0x7f5ddc10ffb0 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5dc40396f0 msgr2=0x7f5dc403bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5dc40396f0 0x7f5dc403bbb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8006fd0 tx=0x7f5dc8006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 msgr2=0x7f5ddc1a28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1a28b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000b7b0 tx=0x7f5dd000bd30 comp rx=0 tx=0).stop 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 shutdown_connections 2024-04-26T01:45:52.500 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5dc40396f0 0x7f5dc403bbb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:52.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 --2- 172.21.15.37:0/3049362400 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ddc10ffb0 0x7f5ddc1a28b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:52.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 >> 172.21.15.37:0/3049362400 conn(0x7f5ddc07dbf0 msgr2=0x7f5ddc07f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:52.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 shutdown_connections 2024-04-26T01:45:52.501 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:51.755+0000 7f5de2200700 1 -- 172.21.15.37:0/3049362400 wait complete. 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.140+0000 7fe2ab800700 1 Processor -- start 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.140+0000 7fe2ab800700 1 -- start start 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.141+0000 7fe2ab800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a41089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.141+0000 7fe2ab800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a4108fb0 con 0x7fe2a41085c0 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.141+0000 7fe2a8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a41089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.141+0000 7fe2a8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a41089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38972/0 (socket says 172.21.15.37:38972) 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.141+0000 7fe2a8e00700 1 -- 172.21.15.37:0/1667127184 learned_addr learned my addr 172.21.15.37:0/1667127184 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.142+0000 7fe2a8e00700 1 -- 172.21.15.37:0/1667127184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a41097c0 con 0x7fe2a41085c0 2024-04-26T01:45:54.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.142+0000 7fe2a8e00700 1 --2- 172.21.15.37:0/1667127184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a41089e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2980098a0 tx=0x7fe298009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5826d03bd7c8465d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.142+0000 7fe2a3400700 1 -- 172.21.15.37:0/1667127184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe298004160 con 0x7fe2a41085c0 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2a3400700 1 -- 172.21.15.37:0/1667127184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe298004300 con 0x7fe2a41085c0 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2a3400700 1 -- 172.21.15.37:0/1667127184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe298011a20 con 0x7fe2a41085c0 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2ab800700 1 -- 172.21.15.37:0/1667127184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 msgr2=0x7fe2a41089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2ab800700 1 --2- 172.21.15.37:0/1667127184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a41089e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2980098a0 tx=0x7fe298009bb0 comp rx=0 tx=0).stop 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2ab800700 1 -- 172.21.15.37:0/1667127184 shutdown_connections 2024-04-26T01:45:54.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2ab800700 1 --2- 172.21.15.37:0/1667127184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a41089e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.143+0000 7fe2ab800700 1 -- 172.21.15.37:0/1667127184 >> 172.21.15.37:0/1667127184 conn(0x7fe2a4103be0 msgr2=0x7fe2a4106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.144+0000 7fe2ab800700 1 -- 172.21.15.37:0/1667127184 shutdown_connections 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.144+0000 7fe2ab800700 1 -- 172.21.15.37:0/1667127184 wait complete. 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.145+0000 7fe2ab800700 1 Processor -- start 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.145+0000 7fe2ab800700 1 -- start start 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.145+0000 7fe2ab800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:54.213 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.145+0000 7fe2ab800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a410cb70 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.145+0000 7fe2a8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.145+0000 7fe2a8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38980/0 (socket says 172.21.15.37:38980) 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.146+0000 7fe2a8e00700 1 -- 172.21.15.37:0/1540134874 learned_addr learned my addr 172.21.15.37:0/1540134874 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.146+0000 7fe2a8e00700 1 -- 172.21.15.37:0/1540134874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a419ebc0 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.146+0000 7fe2a8e00700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a419e680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe2980116e0 tx=0x7fe298011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.146+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe2980039e0 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.147+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe29801b8f0 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.147+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a419ee50 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.147+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe298023760 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.147+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a41a1b40 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.148+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe29801b440 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.148+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2a410ccf0 con 0x7fe2a41085c0 2024-04-26T01:45:54.214 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.148+0000 7fe2a1600700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe28c039650 0x7fe28c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.148+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe29804e1b0 con 0x7fe2a41085c0 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.149+0000 7fe2a3e00700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe28c039650 0x7fe28c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.150+0000 7fe2a3e00700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe28c039650 0x7fe28c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe290006fd0 tx=0x7fe290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.156+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe29801f030 con 0x7fe2a41085c0 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.510+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe2a41a1d10 con 0x7fe2a41085c0 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.515+0000 7fe2a1600700 1 -- 172.21.15.37:0/1540134874 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe29802bb90 con 0x7fe2a41085c0 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.518+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe28c039650 msgr2=0x7fe28c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.518+0000 7fe2ab800700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe28c039650 0x7fe28c03bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe290006fd0 tx=0x7fe290006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.518+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 msgr2=0x7fe2a419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.518+0000 7fe2ab800700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a419e680 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe2980116e0 tx=0x7fe298011aa0 comp rx=0 tx=0).stop 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.519+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 shutdown_connections 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.519+0000 7fe2ab800700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe28c039650 0x7fe28c03bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.519+0000 7fe2ab800700 1 --2- 172.21.15.37:0/1540134874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe2a41085c0 0x7fe2a419e680 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:54.215 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.519+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 >> 172.21.15.37:0/1540134874 conn(0x7fe2a4103be0 msgr2=0x7fe2a4104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.519+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 shutdown_connections 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-04-26T01:45:53.519+0000 7fe2ab800700 1 -- 172.21.15.37:0/1540134874 wait complete. 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.216 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:45:54.217 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-04-26T01:45:54.284 INFO:tasks.cephadm:Fetching config... 2024-04-26T01:45:54.284 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:45:54.284 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T01:45:54.330 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T01:45:54.331 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:45:54.331 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T01:45:54.417 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T01:45:54.418 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:45:54.418 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T01:45:54.537 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T01:45:54.537 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:45:54.537 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T01:45:54.631 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T01:45:54.631 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:45:54.747 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:45:54.771 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:54 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1540134874' entity='client.admin' 2024-04-26T01:45:54.790 DEBUG:teuthology.orchestra.run.smithi045:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:45:54.831 INFO:teuthology.orchestra.run.smithi045.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:45:54.843 DEBUG:teuthology.orchestra.run.smithi071:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T01:45:54.882 INFO:teuthology.orchestra.run.smithi071.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCfwgeiGABFY1R4uwzaC0A2EMGnaHOeK4jXWuLqqhDoIRVBXPrZScRJiadvzhWqBV1bokC7VMApk1khxM9A5LpFETOVrM01ZYcopfOhkrk7SEAAx2I8DzQptuwmbtZDjjTZDJBlpRlGiuKto+Z4E5uKlp1BJrIJKNsC1B5jKp2YWGjc3rfkUapqRkJtHllOsOimiLzvBnErc3xMI/MIFl/20KmHPk1ZtGup3ON2jnO2TXFIL2byNNfS2WGnuKuMe4B1NopHhicmEdHsowdt3GuYEdmLZbgo0bNTY69UF1V1Pd26vYlQ9GNWiqXVh8pA4iEscuv//RFpWiy3ln7LxDEXVY1DPotmuleN8xWb+lbXbInDyU9UUZiFmMgZcleaNtaa8jmFxEt6Dh+kxRhmDoprxVYAlo37V3UUpjY4GXZDLagLwkTcfxkD8WcHQaDrfM73DXsssekQyGgtQg6+jsPwS6CWc+rmI6vFSMomihN2mrgunTskSUfe9SjFduhWpWM= ceph-5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:45:54.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T01:45:55.333 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 -- 172.21.15.37:0/2158564868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c102f30 msgr2=0x7fdc0c105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 --2- 172.21.15.37:0/2158564868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c102f30 0x7fdc0c105320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009910 tx=0x7fdc00009c20 comp rx=0 tx=0).stop 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 -- 172.21.15.37:0/2158564868 shutdown_connections 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 --2- 172.21.15.37:0/2158564868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c102f30 0x7fdc0c105320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 -- 172.21.15.37:0/2158564868 >> 172.21.15.37:0/2158564868 conn(0x7fdc0c0fc940 msgr2=0x7fdc0c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 -- 172.21.15.37:0/2158564868 shutdown_connections 2024-04-26T01:45:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.931+0000 7fdc13800700 1 -- 172.21.15.37:0/2158564868 wait complete. 2024-04-26T01:45:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.933+0000 7fdc13800700 1 Processor -- start 2024-04-26T01:45:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.933+0000 7fdc13800700 1 -- start start 2024-04-26T01:45:57.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.934+0000 7fdc13800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 0x7fdc0c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:57.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.934+0000 7fdc13800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c108380 con 0x7fdc0c197c60 2024-04-26T01:45:57.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.934+0000 7fdc10e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 0x7fdc0c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.934+0000 7fdc10e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 0x7fdc0c198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39000/0 (socket says 172.21.15.37:39000) 2024-04-26T01:45:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.934+0000 7fdc10e00700 1 -- 172.21.15.37:0/4263925123 learned_addr learned my addr 172.21.15.37:0/4263925123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:45:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.934+0000 7fdc10e00700 1 -- 172.21.15.37:0/4263925123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c1985c0 con 0x7fdc0c197c60 2024-04-26T01:45:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.935+0000 7fdc10e00700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 0x7fdc0c198080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.935+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc00026070 con 0x7fdc0c197c60 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.935+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc0c1988b0 con 0x7fdc0c197c60 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.935+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc0000bba0 con 0x7fdc0c197c60 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.935+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc000038a0 con 0x7fdc0c197c60 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.935+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc0c19b3c0 con 0x7fdc0c197c60 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.936+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc0c108b00 con 0x7fdc0c197c60 2024-04-26T01:45:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.936+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdc00029070 con 0x7fdc0c197c60 2024-04-26T01:45:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.937+0000 7fdc09600700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdbf40396b0 0x7fdbf403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:45:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.937+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fdc0001e070 con 0x7fdc0c197c60 2024-04-26T01:45:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.937+0000 7fdc0be00700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdbf40396b0 0x7fdbf403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:45:57.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.938+0000 7fdc0be00700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdbf40396b0 0x7fdbf403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006fd0 tx=0x7fdbf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:45:57.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:57.943+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc00013070 con 0x7fdc0c197c60 2024-04-26T01:45:58.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.182+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fdc0c02e020 con 0x7fdc0c197c60 2024-04-26T01:45:58.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.186+0000 7fdc09600700 1 -- 172.21.15.37:0/4263925123 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fdc000300c0 con 0x7fdc0c197c60 2024-04-26T01:45:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdbf40396b0 msgr2=0x7fdbf403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:58.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdbf40396b0 0x7fdbf403bb70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8006fd0 tx=0x7fdbf8006e40 comp rx=0 tx=0).stop 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 msgr2=0x7fdc0c198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 0x7fdc0c198080 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00005ad0 comp rx=0 tx=0).stop 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 shutdown_connections 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdbf40396b0 0x7fdbf403bb70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 --2- 172.21.15.37:0/4263925123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdc0c197c60 0x7fdc0c198080 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 >> 172.21.15.37:0/4263925123 conn(0x7fdc0c0fc940 msgr2=0x7fdc0c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 shutdown_connections 2024-04-26T01:45:58.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:45:58.189+0000 7fdc13800700 1 -- 172.21.15.37:0/4263925123 wait complete. 2024-04-26T01:45:59.002 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T01:45:59.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T01:45:59.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:45:59 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4263925123' entity='client.admin' 2024-04-26T01:45:59.553 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:02.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.059+0000 7fab03200700 1 -- 172.21.15.37:0/1574839876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 msgr2=0x7faafc105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:02.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.059+0000 7fab03200700 1 --2- 172.21.15.37:0/1574839876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc105360 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7faaec009910 tx=0x7faaec009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:02.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.059+0000 7fab03200700 1 -- 172.21.15.37:0/1574839876 shutdown_connections 2024-04-26T01:46:02.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.059+0000 7fab03200700 1 --2- 172.21.15.37:0/1574839876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc105360 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.059+0000 7fab03200700 1 -- 172.21.15.37:0/1574839876 >> 172.21.15.37:0/1574839876 conn(0x7faafc0fc940 msgr2=0x7faafc0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:02.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.060+0000 7fab03200700 1 -- 172.21.15.37:0/1574839876 shutdown_connections 2024-04-26T01:46:02.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.060+0000 7fab03200700 1 -- 172.21.15.37:0/1574839876 wait complete. 2024-04-26T01:46:02.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.061+0000 7fab03200700 1 Processor -- start 2024-04-26T01:46:02.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.061+0000 7fab03200700 1 -- start start 2024-04-26T01:46:02.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.062+0000 7fab03200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc1027a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:02.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.062+0000 7fab03200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faafc1083e0 con 0x7faafc102f70 2024-04-26T01:46:02.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.062+0000 7faafbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc1027a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:02.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.062+0000 7faafbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc1027a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41656/0 (socket says 172.21.15.37:41656) 2024-04-26T01:46:02.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.062+0000 7faafbe00700 1 -- 172.21.15.37:0/1237995000 learned_addr learned my addr 172.21.15.37:0/1237995000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:02.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.062+0000 7faafbe00700 1 -- 172.21.15.37:0/1237995000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc102ce0 con 0x7faafc102f70 2024-04-26T01:46:02.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.063+0000 7faafbe00700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc1027a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faaec005190 tx=0x7faaec004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:02.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.063+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaec027070 con 0x7faafc102f70 2024-04-26T01:46:02.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.063+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaec022470 con 0x7faafc102f70 2024-04-26T01:46:02.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.063+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faaec00f5f0 con 0x7faafc102f70 2024-04-26T01:46:02.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.063+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc100f20 con 0x7faafc102f70 2024-04-26T01:46:02.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.063+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc101400 con 0x7faafc102f70 2024-04-26T01:46:02.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.065+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7faaec02a070 con 0x7faafc102f70 2024-04-26T01:46:02.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.065+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faafc108b60 con 0x7faafc102f70 2024-04-26T01:46:02.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.065+0000 7faaf8c00700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faae00396b0 0x7faae003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:02.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.065+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7faaec01e030 con 0x7faafc102f70 2024-04-26T01:46:02.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.066+0000 7faafb400700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faae00396b0 0x7faae003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:02.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.067+0000 7faafb400700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faae00396b0 0x7faae003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae8006fd0 tx=0x7faae8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:02.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.071+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faaec013070 con 0x7faafc102f70 2024-04-26T01:46:02.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.323+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7faafc190e40 con 0x7faae00396b0 2024-04-26T01:46:02.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.329+0000 7faaf8c00700 1 -- 172.21.15.37:0/1237995000 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7faafc190e40 con 0x7faae00396b0 2024-04-26T01:46:02.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faae00396b0 msgr2=0x7faae003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:02.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faae00396b0 0x7faae003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7faae8006fd0 tx=0x7faae8006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:02.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 msgr2=0x7faafc1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc1027a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7faaec005190 tx=0x7faaec004f80 comp rx=0 tx=0).stop 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 shutdown_connections 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faae00396b0 0x7faae003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 --2- 172.21.15.37:0/1237995000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faafc102f70 0x7faafc1027a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 >> 172.21.15.37:0/1237995000 conn(0x7faafc0fc940 msgr2=0x7faafc0fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 shutdown_connections 2024-04-26T01:46:02.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:02.332+0000 7fab03200700 1 -- 172.21.15.37:0/1237995000 wait complete. 2024-04-26T01:46:03.187 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi045 2024-04-26T01:46:03.187 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:46:03.187 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/etc/ceph/ceph.conf 2024-04-26T01:46:03.208 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:46:03.208 DEBUG:teuthology.orchestra.run.smithi045:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:03.266 INFO:tasks.cephadm:Adding host smithi045 to orchestrator... 2024-04-26T01:46:03.266 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch host add smithi045 2024-04-26T01:46:03.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:03.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:03.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:03.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:03.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:03.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:03.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:03.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:03.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:03.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:03.702 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:04.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:04 smithi037 ceph-mon[27906]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:04.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:04 smithi037 ceph-mon[27906]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:46:04.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:04 smithi037 ceph-mon[27906]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:04.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:04 smithi037 ceph-mon[27906]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:05.319 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:05 smithi037 ceph-mon[27906]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:05.319 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:05.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:05.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:05.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.835+0000 7f2053e00700 1 -- 172.21.15.37:0/1291851386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 msgr2=0x7f2054075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:05.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.836+0000 7f2053e00700 1 --2- 172.21.15.37:0/1291851386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054075210 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f204400b0a0 tx=0x7f204400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:46:05.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.836+0000 7f2053e00700 1 -- 172.21.15.37:0/1291851386 shutdown_connections 2024-04-26T01:46:05.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.836+0000 7f2053e00700 1 --2- 172.21.15.37:0/1291851386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054075210 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:05.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.836+0000 7f2053e00700 1 -- 172.21.15.37:0/1291851386 >> 172.21.15.37:0/1291851386 conn(0x7f20540fdda0 msgr2=0x7f20541001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:05.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.836+0000 7f2053e00700 1 -- 172.21.15.37:0/1291851386 shutdown_connections 2024-04-26T01:46:05.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.836+0000 7f2053e00700 1 -- 172.21.15.37:0/1291851386 wait complete. 2024-04-26T01:46:05.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.837+0000 7f2053e00700 1 Processor -- start 2024-04-26T01:46:05.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.837+0000 7f2053e00700 1 -- start start 2024-04-26T01:46:05.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.838+0000 7f2053e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:05.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.838+0000 7f2053e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f205410e9e0 con 0x7f2054074e30 2024-04-26T01:46:05.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.838+0000 7f2052a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.838+0000 7f2052a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41684/0 (socket says 172.21.15.37:41684) 2024-04-26T01:46:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.838+0000 7f2052a00700 1 -- 172.21.15.37:0/2653043388 learned_addr learned my addr 172.21.15.37:0/2653043388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.838+0000 7f2052a00700 1 -- 172.21.15.37:0/2653043388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f205419a3e0 con 0x7f2054074e30 2024-04-26T01:46:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.839+0000 7f2052a00700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054199ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f204400d010 tx=0x7f2044009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:05.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.839+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2044021020 con 0x7f2054074e30 2024-04-26T01:46:05.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.839+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2044004000 con 0x7f2054074e30 2024-04-26T01:46:05.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.839+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2044017040 con 0x7f2054074e30 2024-04-26T01:46:05.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.839+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205419a670 con 0x7f2054074e30 2024-04-26T01:46:05.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.839+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205419e770 con 0x7f2054074e30 2024-04-26T01:46:05.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.840+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2044024020 con 0x7f2054074e30 2024-04-26T01:46:05.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.841+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20540764e0 con 0x7f2054074e30 2024-04-26T01:46:05.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.841+0000 7f2058a00700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f20380396f0 0x7f203803bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:05.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.841+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f204400f050 con 0x7f2054074e30 2024-04-26T01:46:05.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.841+0000 7f204a000700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f20380396f0 0x7f203803bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:05.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.842+0000 7f204a000700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f20380396f0 0x7f203803bbb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f203c006fd0 tx=0x7f203c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:05.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:05.847+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f204402b020 con 0x7f2054074e30 2024-04-26T01:46:06.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:06.083+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi045", "target": ["mon-mgr", ""]}) -- 0x7f20540640a0 con 0x7f20380396f0 2024-04-26T01:46:06.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:06 smithi037 ceph-mon[27906]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:46:06.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:06 smithi037 ceph-mon[27906]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:07.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:07 smithi037 ceph-mon[27906]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi045", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:08.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:08 smithi037 ceph-mon[27906]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:08.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:08 smithi037 ceph-mon[27906]: Deploying cephadm binary to smithi045 2024-04-26T01:46:10.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:10 smithi037 ceph-mon[27906]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:11.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.081+0000 7f2058a00700 1 -- 172.21.15.37:0/2653043388 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f20540640a0 con 0x7f20380396f0 2024-04-26T01:46:11.079 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi045' with addr '172.21.15.45' 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.084+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f20380396f0 msgr2=0x7f203803bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.084+0000 7f2053e00700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f20380396f0 0x7f203803bbb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f203c006fd0 tx=0x7f203c006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.084+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 msgr2=0x7f2054199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.084+0000 7f2053e00700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054199ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f204400d010 tx=0x7f2044009790 comp rx=0 tx=0).stop 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.085+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 shutdown_connections 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.085+0000 7f2053e00700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f20380396f0 0x7f203803bbb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.085+0000 7f2053e00700 1 --2- 172.21.15.37:0/2653043388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2054074e30 0x7f2054199ea0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:11.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.085+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 >> 172.21.15.37:0/2653043388 conn(0x7f20540fdda0 msgr2=0x7f2054106570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:11.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.085+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 shutdown_connections 2024-04-26T01:46:11.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:11.085+0000 7f2053e00700 1 -- 172.21.15.37:0/2653043388 wait complete. 2024-04-26T01:46:11.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T01:46:12.197 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:12.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:12 smithi037 ceph-mon[27906]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:12.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:12.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:12 smithi037 ceph-mon[27906]: Added host smithi045 2024-04-26T01:46:12.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:12.349 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:13.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:13.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:13 smithi037 ceph-mon[27906]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:14.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 -- 172.21.15.37:0/3177891572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 msgr2=0x7ff51810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 --2- 172.21.15.37:0/3177891572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51810dd40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff508009910 tx=0x7ff508009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 -- 172.21.15.37:0/3177891572 shutdown_connections 2024-04-26T01:46:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 --2- 172.21.15.37:0/3177891572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51810dd40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 -- 172.21.15.37:0/3177891572 >> 172.21.15.37:0/3177891572 conn(0x7ff5180781e0 msgr2=0x7ff51807a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 -- 172.21.15.37:0/3177891572 shutdown_connections 2024-04-26T01:46:14.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.458+0000 7ff51f400700 1 -- 172.21.15.37:0/3177891572 wait complete. 2024-04-26T01:46:14.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.460+0000 7ff51f400700 1 Processor -- start 2024-04-26T01:46:14.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.460+0000 7ff51f400700 1 -- start start 2024-04-26T01:46:14.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.460+0000 7ff51f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:14.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.460+0000 7ff51f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff518112f60 con 0x7ff51810d960 2024-04-26T01:46:14.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.461+0000 7ff51ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:14.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.461+0000 7ff51ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51819e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43640/0 (socket says 172.21.15.37:43640) 2024-04-26T01:46:14.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.461+0000 7ff51ca00700 1 -- 172.21.15.37:0/3925301193 learned_addr learned my addr 172.21.15.37:0/3925301193 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:14.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.461+0000 7ff51ca00700 1 -- 172.21.15.37:0/3925301193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff51819e9b0 con 0x7ff51810d960 2024-04-26T01:46:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.461+0000 7ff51ca00700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51819e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff508006010 tx=0x7ff508005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.462+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff508026070 con 0x7ff51810d960 2024-04-26T01:46:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.462+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50800bba0 con 0x7ff51810d960 2024-04-26T01:46:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.462+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff508003930 con 0x7ff51810d960 2024-04-26T01:46:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.462+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff51819eca0 con 0x7ff51810d960 2024-04-26T01:46:14.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.462+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5181a2d40 con 0x7ff51810d960 2024-04-26T01:46:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.463+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff508029070 con 0x7ff51810d960 2024-04-26T01:46:14.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.463+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff51810a180 con 0x7ff51810d960 2024-04-26T01:46:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.464+0000 7ff516000700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff4fc0396b0 0x7ff4fc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.464+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff50801e070 con 0x7ff51810d960 2024-04-26T01:46:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.464+0000 7ff50fe00700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff4fc0396b0 0x7ff4fc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:14.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.465+0000 7ff50fe00700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff4fc0396b0 0x7ff4fc03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff500006fd0 tx=0x7ff500006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:14.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.470+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff508013070 con 0x7ff51810d960 2024-04-26T01:46:14.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.704+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff51819f7d0 con 0x7ff4fc0396b0 2024-04-26T01:46:14.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.707+0000 7ff516000700 1 -- 172.21.15.37:0/3925301193 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7ff51819f7d0 con 0x7ff4fc0396b0 2024-04-26T01:46:14.705 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:46:14.705 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}] 2024-04-26T01:46:14.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.710+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff4fc0396b0 msgr2=0x7ff4fc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:14.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.710+0000 7ff51f400700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff4fc0396b0 0x7ff4fc03bb70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff500006fd0 tx=0x7ff500006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:14.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.710+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 msgr2=0x7ff51819e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:14.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.710+0000 7ff51f400700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51819e470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff508006010 tx=0x7ff508005780 comp rx=0 tx=0).stop 2024-04-26T01:46:14.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.711+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 shutdown_connections 2024-04-26T01:46:14.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.711+0000 7ff51f400700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff4fc0396b0 0x7ff4fc03bb70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:14.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.711+0000 7ff51f400700 1 --2- 172.21.15.37:0/3925301193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff51810d960 0x7ff51819e470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:14.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.711+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 >> 172.21.15.37:0/3925301193 conn(0x7ff5180781e0 msgr2=0x7ff518078f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:14.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.711+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 shutdown_connections 2024-04-26T01:46:14.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:14.711+0000 7ff51f400700 1 -- 172.21.15.37:0/3925301193 wait complete. 2024-04-26T01:46:14.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:15.423 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi071 2024-04-26T01:46:15.423 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:46:15.423 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.conf 2024-04-26T01:46:15.441 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:46:15.441 DEBUG:teuthology.orchestra.run.smithi071:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:15.498 INFO:tasks.cephadm:Adding host smithi071 to orchestrator... 2024-04-26T01:46:15.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch host add smithi071 2024-04-26T01:46:15.961 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:16.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:15 smithi037 ceph-mon[27906]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:16.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:15 smithi037 ceph-mon[27906]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 -- 172.21.15.37:0/1527717008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 msgr2=0x7f6a5c0751c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 --2- 172.21.15.37:0/1527717008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c0751c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c009910 tx=0x7f6a4c009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 -- 172.21.15.37:0/1527717008 shutdown_connections 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 --2- 172.21.15.37:0/1527717008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c0751c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 -- 172.21.15.37:0/1527717008 >> 172.21.15.37:0/1527717008 conn(0x7f6a5c0fdf80 msgr2=0x7f6a5c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 -- 172.21.15.37:0/1527717008 shutdown_connections 2024-04-26T01:46:18.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.053+0000 7f6a62600700 1 -- 172.21.15.37:0/1527717008 wait complete. 2024-04-26T01:46:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.055+0000 7f6a62600700 1 Processor -- start 2024-04-26T01:46:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.055+0000 7f6a62600700 1 -- start start 2024-04-26T01:46:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.055+0000 7f6a62600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c19e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:18.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.055+0000 7f6a62600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c112fa0 con 0x7f6a5c074de0 2024-04-26T01:46:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.056+0000 7f6a5b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c19e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.056+0000 7f6a5b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c19e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43662/0 (socket says 172.21.15.37:43662) 2024-04-26T01:46:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.056+0000 7f6a5b400700 1 -- 172.21.15.37:0/3786947623 learned_addr learned my addr 172.21.15.37:0/3786947623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:18.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.056+0000 7f6a5b400700 1 -- 172.21.15.37:0/3786947623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c19ea20 con 0x7f6a5c074de0 2024-04-26T01:46:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.056+0000 7f6a5b400700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c19e4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c006010 tx=0x7f6a4c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.057+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c026070 con 0x7f6a5c074de0 2024-04-26T01:46:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.057+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a4c00bc00 con 0x7f6a5c074de0 2024-04-26T01:46:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.057+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c19ecb0 con 0x7f6a5c074de0 2024-04-26T01:46:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.057+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a4c003910 con 0x7f6a5c074de0 2024-04-26T01:46:18.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.057+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c19f110 con 0x7f6a5c074de0 2024-04-26T01:46:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.058+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6a4c029070 con 0x7f6a5c074de0 2024-04-26T01:46:18.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.058+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a5c076520 con 0x7f6a5c074de0 2024-04-26T01:46:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.058+0000 7f6a58c00700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6a400396b0 0x7f6a4003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.059+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6a4c01e070 con 0x7f6a5c074de0 2024-04-26T01:46:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.059+0000 7f6a52a00700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6a400396b0 0x7f6a4003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:18.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.059+0000 7f6a52a00700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6a400396b0 0x7f6a4003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400ad30 tx=0x7f6a440093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:18.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.064+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a4c013070 con 0x7f6a5c074de0 2024-04-26T01:46:18.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:18 smithi037 ceph-mon[27906]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:46:18.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:18.303+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}) -- 0x7f6a5c19f8b0 con 0x7f6a400396b0 2024-04-26T01:46:19.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:19 smithi037 ceph-mon[27906]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:19.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:19 smithi037 ceph-mon[27906]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:19.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:19 smithi037 ceph-mon[27906]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:46:19.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:19.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:19.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:20.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:20 smithi037 ceph-mon[27906]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:20.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:20 smithi037 ceph-mon[27906]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:20.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:20 smithi037 ceph-mon[27906]: Deploying cephadm binary to smithi071 2024-04-26T01:46:22.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:22 smithi037 ceph-mon[27906]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:23.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.273+0000 7f6a58c00700 1 -- 172.21.15.37:0/3786947623 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f6a5c19f8b0 con 0x7f6a400396b0 2024-04-26T01:46:23.273 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi071' with addr '172.21.15.71' 2024-04-26T01:46:23.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6a400396b0 msgr2=0x7f6a4003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6a400396b0 0x7f6a4003bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6a4400ad30 tx=0x7f6a440093f0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 msgr2=0x7f6a5c19e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:23.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c19e4e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c006010 tx=0x7f6a4c005780 comp rx=0 tx=0).stop 2024-04-26T01:46:23.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 shutdown_connections 2024-04-26T01:46:23.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6a400396b0 0x7f6a4003bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 --2- 172.21.15.37:0/3786947623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c074de0 0x7f6a5c19e4e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:23.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 >> 172.21.15.37:0/3786947623 conn(0x7f6a5c0fdf80 msgr2=0x7f6a5c1041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:23.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 shutdown_connections 2024-04-26T01:46:23.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:23.277+0000 7f6a62600700 1 -- 172.21.15.37:0/3786947623 wait complete. 2024-04-26T01:46:24.066 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T01:46:24.456 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:24 smithi037 ceph-mon[27906]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:24.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:24.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:24.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:24.522 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:25.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:25 smithi037 ceph-mon[27906]: Added host smithi071 2024-04-26T01:46:25.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:26.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:26 smithi037 ceph-mon[27906]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:26.372 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:26 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 -- 172.21.15.37:0/1250976036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 msgr2=0x7f0d001028d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 --2- 172.21.15.37:0/1250976036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d001028d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0cf4009910 tx=0x7f0cf4009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 -- 172.21.15.37:0/1250976036 shutdown_connections 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 --2- 172.21.15.37:0/1250976036 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d001028d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 -- 172.21.15.37:0/1250976036 >> 172.21.15.37:0/1250976036 conn(0x7f0d000fdda0 msgr2=0x7f0d001001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 -- 172.21.15.37:0/1250976036 shutdown_connections 2024-04-26T01:46:26.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.672+0000 7f0d06200700 1 -- 172.21.15.37:0/1250976036 wait complete. 2024-04-26T01:46:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.673+0000 7f0d06200700 1 Processor -- start 2024-04-26T01:46:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.673+0000 7f0d06200700 1 -- start start 2024-04-26T01:46:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.674+0000 7f0d06200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d0019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.674+0000 7f0d06200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d0010c770 con 0x7f0d001024f0 2024-04-26T01:46:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.674+0000 7f0d04e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d0019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:26.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.674+0000 7f0d04e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d0019da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50904/0 (socket says 172.21.15.37:50904) 2024-04-26T01:46:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.674+0000 7f0d04e00700 1 -- 172.21.15.37:0/3163736306 learned_addr learned my addr 172.21.15.37:0/3163736306 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0d04e00700 1 -- 172.21.15.37:0/3163736306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0019dfa0 con 0x7f0d001024f0 2024-04-26T01:46:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0d04e00700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d0019da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40056d0 tx=0x7f0cf40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cf4026070 con 0x7f0d001024f0 2024-04-26T01:46:26.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cf400bba0 con 0x7f0d001024f0 2024-04-26T01:46:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cf4003920 con 0x7f0d001024f0 2024-04-26T01:46:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0019e290 con 0x7f0d001024f0 2024-04-26T01:46:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.675+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d00197d00 con 0x7f0d001024f0 2024-04-26T01:46:26.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.676+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d001039d0 con 0x7f0d001024f0 2024-04-26T01:46:26.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.677+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0cf4029070 con 0x7f0d001024f0 2024-04-26T01:46:26.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.677+0000 7f0cfd600700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f0ce8039650 0x7f0ce803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:26.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.677+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0cf401e070 con 0x7f0d001024f0 2024-04-26T01:46:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.677+0000 7f0cffe00700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f0ce8039650 0x7f0ce803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:26.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.678+0000 7f0cffe00700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f0ce8039650 0x7f0ce803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006fd0 tx=0x7f0cec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:26.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.683+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0cf4013070 con 0x7f0d001024f0 2024-04-26T01:46:26.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.919+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0d00199060 con 0x7f0ce8039650 2024-04-26T01:46:26.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.921+0000 7f0cfd600700 1 -- 172.21.15.37:0/3163736306 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+238 (secure 0 0 0) 0x7f0d00199060 con 0x7f0ce8039650 2024-04-26T01:46:26.919 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:46:26.920 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.45", "hostname": "smithi045", "labels": [], "status": ""}, {"addr": "172.21.15.71", "hostname": "smithi071", "labels": [], "status": ""}] 2024-04-26T01:46:26.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f0ce8039650 msgr2=0x7f0ce803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f0ce8039650 0x7f0ce803bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cec006fd0 tx=0x7f0cec006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 msgr2=0x7f0d0019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d0019da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0cf40056d0 tx=0x7f0cf40057b0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 shutdown_connections 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f0ce8039650 0x7f0ce803bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 --2- 172.21.15.37:0/3163736306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d001024f0 0x7f0d0019da60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 >> 172.21.15.37:0/3163736306 conn(0x7f0d000fdda0 msgr2=0x7f0d00103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.924+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 shutdown_connections 2024-04-26T01:46:26.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:26.925+0000 7f0d06200700 1 -- 172.21.15.37:0/3163736306 wait complete. 2024-04-26T01:46:27.712 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T01:46:27.713 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T01:46:28.159 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:28.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:28 smithi037 ceph-mon[27906]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:28.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:28 smithi037 ceph-mon[27906]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 -- 172.21.15.37:0/2665570566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 msgr2=0x7f98a410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 --2- 172.21.15.37:0/2665570566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a410dd40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f989800b0a0 tx=0x7f989800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 -- 172.21.15.37:0/2665570566 shutdown_connections 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 --2- 172.21.15.37:0/2665570566 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a410dd40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 -- 172.21.15.37:0/2665570566 >> 172.21.15.37:0/2665570566 conn(0x7f98a40781e0 msgr2=0x7f98a407a600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 -- 172.21.15.37:0/2665570566 shutdown_connections 2024-04-26T01:46:30.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.244+0000 7f98abe00700 1 -- 172.21.15.37:0/2665570566 wait complete. 2024-04-26T01:46:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.245+0000 7f98abe00700 1 Processor -- start 2024-04-26T01:46:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.246+0000 7f98abe00700 1 -- start start 2024-04-26T01:46:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.246+0000 7f98abe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a419e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:30.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.246+0000 7f98abe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4112f60 con 0x7f98a410d960 2024-04-26T01:46:30.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.247+0000 7f98a9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a419e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:30.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.247+0000 7f98a9400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a419e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38610/0 (socket says 172.21.15.37:38610) 2024-04-26T01:46:30.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.247+0000 7f98a9400700 1 -- 172.21.15.37:0/1253064705 learned_addr learned my addr 172.21.15.37:0/1253064705 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:30.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.247+0000 7f98a9400700 1 -- 172.21.15.37:0/1253064705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a419e8f0 con 0x7f98a410d960 2024-04-26T01:46:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.248+0000 7f98a9400700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a419e3b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f989800d010 tx=0x7f9898009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.248+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9898021020 con 0x7f98a410d960 2024-04-26T01:46:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.248+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a419eb80 con 0x7f98a410d960 2024-04-26T01:46:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.248+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9898004000 con 0x7f98a410d960 2024-04-26T01:46:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.248+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9898017040 con 0x7f98a410d960 2024-04-26T01:46:30.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.248+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a2c80 con 0x7f98a410d960 2024-04-26T01:46:30.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.249+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a410a180 con 0x7f98a410d960 2024-04-26T01:46:30.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.250+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9898024020 con 0x7f98a410d960 2024-04-26T01:46:30.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.250+0000 7f98a2000700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f988c0396a0 0x7f988c03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:30.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.250+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f989800f050 con 0x7f98a410d960 2024-04-26T01:46:30.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.251+0000 7f98a8a00700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f988c0396a0 0x7f988c03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:30.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.251+0000 7f98a8a00700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f988c0396a0 0x7f988c03bb60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9890006fd0 tx=0x7f9890006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:30.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.256+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f989802b020 con 0x7f98a410d960 2024-04-26T01:46:30.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:30.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:30.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:30.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:30.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:30.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:30.498+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f98a40640a0 con 0x7f98a410d960 2024-04-26T01:46:31.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.119+0000 7f98a2000700 1 -- 172.21.15.37:0/1253064705 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f9898011390 con 0x7f98a410d960 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f988c0396a0 msgr2=0x7f988c03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f988c0396a0 0x7f988c03bb60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9890006fd0 tx=0x7f9890006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 msgr2=0x7f98a419e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a419e3b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f989800d010 tx=0x7f9898009790 comp rx=0 tx=0).stop 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 shutdown_connections 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f988c0396a0 0x7f988c03bb60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 --2- 172.21.15.37:0/1253064705 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98a410d960 0x7f98a419e3b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:31.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.123+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 >> 172.21.15.37:0/1253064705 conn(0x7f98a40781e0 msgr2=0x7f98a41104f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:31.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.124+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 shutdown_connections 2024-04-26T01:46:31.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:31.124+0000 7f98abe00700 1 -- 172.21.15.37:0/1253064705 wait complete. 2024-04-26T01:46:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-04-26T01:46:31.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:46:31.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:31.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:31.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1253064705' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T01:46:31.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:31.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:31.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:31.893 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-04-26T01:46:31.893 INFO:tasks.cephadm:Adding mon.b on smithi045 2024-04-26T01:46:31.893 INFO:tasks.cephadm:Adding mon.c on smithi071 2024-04-26T01:46:31.893 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c' 2024-04-26T01:46:32.129 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:32.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:32 smithi037 ceph-mon[27906]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:46:32.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:32 smithi037 ceph-mon[27906]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:32.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:32 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1253064705' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T01:46:32.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:32 smithi037 ceph-mon[27906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.604+0000 7f9606ffd700 1 -- 172.21.15.71:0/1443163268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f8005460 con 0x7f9608101590 2024-04-26T01:46:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 -- 172.21.15.71:0/1443163268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 msgr2=0x7f9608101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 --2- 172.21.15.71:0/1443163268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608101970 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f95f8009910 tx=0x7f95f8009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 -- 172.21.15.71:0/1443163268 shutdown_connections 2024-04-26T01:46:33.603 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 --2- 172.21.15.71:0/1443163268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608101970 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 -- 172.21.15.71:0/1443163268 >> 172.21.15.71:0/1443163268 conn(0x7f96080fce40 msgr2=0x7f96080ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 -- 172.21.15.71:0/1443163268 shutdown_connections 2024-04-26T01:46:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.605+0000 7f960e79c700 1 -- 172.21.15.71:0/1443163268 wait complete. 2024-04-26T01:46:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f960e79c700 1 Processor -- start 2024-04-26T01:46:33.604 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f960e79c700 1 -- start start 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f960e79c700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f960e79c700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f960806aec0 con 0x7f9608101590 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f9607fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f9607fff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.71:49334/0 (socket says 172.21.15.71:49334) 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.606+0000 7f9607fff700 1 -- 172.21.15.71:0/1210204013 learned_addr learned my addr 172.21.15.71:0/1210204013 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.607+0000 7f9607fff700 1 -- 172.21.15.71:0/1210204013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96081983d0 con 0x7f9608101590 2024-04-26T01:46:33.605 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.607+0000 7f9607fff700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f95f80056d0 tx=0x7f95f80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:33.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.608+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f8026070 con 0x7f9608101590 2024-04-26T01:46:33.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.608+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96081986c0 con 0x7f9608101590 2024-04-26T01:46:33.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.608+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f960819c760 con 0x7f9608101590 2024-04-26T01:46:33.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.608+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95f800b6f0 con 0x7f9608101590 2024-04-26T01:46:33.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.608+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f95f8030060 con 0x7f9608101590 2024-04-26T01:46:33.606 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.609+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f4005330 con 0x7f9608101590 2024-04-26T01:46:33.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.609+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f95f8029070 con 0x7f9608101590 2024-04-26T01:46:33.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.609+0000 7f96057fa700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f0039630 0x7f95f003baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:33.607 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.609+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f95f801e070 con 0x7f9608101590 2024-04-26T01:46:33.608 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.611+0000 7f96077fe700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f0039630 0x7f95f003baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:33.609 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.612+0000 7f96077fe700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f0039630 0x7f95f003baf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:33.622 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.625+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95f8013070 con 0x7f9608101590 2024-04-26T01:46:33.831 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.833+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}) -- 0x7f95f4000c70 con 0x7f95f0039630 2024-04-26T01:46:33.845 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.847+0000 7f96057fa700 1 -- 172.21.15.71:0/1210204013 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f95f4000c70 con 0x7f95f0039630 2024-04-26T01:46:33.845 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mon update... 2024-04-26T01:46:33.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.849+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f0039630 msgr2=0x7f95f003baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:33.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.849+0000 7f960e79c700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f0039630 0x7f95f003baf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f95fc006fd0 tx=0x7f95fc006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:33.847 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.849+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 msgr2=0x7f9608197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608197e90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f95f80056d0 tx=0x7f95f80057b0 comp rx=0 tx=0).stop 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 shutdown_connections 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f0039630 0x7f95f003baf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 --2- 172.21.15.71:0/1210204013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9608101590 0x7f9608197e90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 >> 172.21.15.71:0/1210204013 conn(0x7f96080fce40 msgr2=0x7f96081056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 shutdown_connections 2024-04-26T01:46:33.848 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:33.850+0000 7f960e79c700 1 -- 172.21.15.71:0/1210204013 wait complete. 2024-04-26T01:46:34.457 DEBUG:teuthology.orchestra.run.smithi045:mon.b> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T01:46:34.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:34.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:34.460 DEBUG:teuthology.orchestra.run.smithi071:mon.c> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T01:46:34.462 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:46:34.463 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:46:35.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:35 smithi037 ceph-mon[27906]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:35.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:35 smithi037 ceph-mon[27906]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c;count:3 2024-04-26T01:46:35.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:35 smithi037 ceph-mon[27906]: Deploying daemon mon.c on smithi071 2024-04-26T01:46:36.136 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:36.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:36 smithi037 ceph-mon[27906]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 -- 172.21.15.71:0/4029262192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 msgr2=0x7fa9c01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 --2- 172.21.15.71:0/4029262192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c01019d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009910 tx=0x7fa9b0009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 -- 172.21.15.71:0/4029262192 shutdown_connections 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 --2- 172.21.15.71:0/4029262192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c01019d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 -- 172.21.15.71:0/4029262192 >> 172.21.15.71:0/4029262192 conn(0x7fa9c0076ea0 msgr2=0x7fa9c00772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 -- 172.21.15.71:0/4029262192 shutdown_connections 2024-04-26T01:46:38.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.016+0000 7fa9c6799700 1 -- 172.21.15.71:0/4029262192 wait complete. 2024-04-26T01:46:38.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.017+0000 7fa9c6799700 1 Processor -- start 2024-04-26T01:46:38.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.017+0000 7fa9c6799700 1 -- start start 2024-04-26T01:46:38.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.018+0000 7fa9c6799700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:38.016 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.018+0000 7fa9c6799700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c010b820 con 0x7fa9c01015f0 2024-04-26T01:46:38.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.018+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:38.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.018+0000 7fa9bffff700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c0197e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.71:49346/0 (socket says 172.21.15.71:49346) 2024-04-26T01:46:38.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.018+0000 7fa9bffff700 1 -- 172.21.15.71:0/1609717789 learned_addr learned my addr 172.21.15.71:0/1609717789 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:46:38.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.019+0000 7fa9bffff700 1 -- 172.21.15.71:0/1609717789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c01983c0 con 0x7fa9c01015f0 2024-04-26T01:46:38.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.019+0000 7fa9bffff700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c0197e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00056d0 tx=0x7fa9b00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:38.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.020+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b0026070 con 0x7fa9c01015f0 2024-04-26T01:46:38.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.020+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c01986b0 con 0x7fa9c01015f0 2024-04-26T01:46:38.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.020+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c019c750 con 0x7fa9c01015f0 2024-04-26T01:46:38.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.020+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b000bba0 con 0x7fa9c01015f0 2024-04-26T01:46:38.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.020+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa9b0030060 con 0x7fa9c01015f0 2024-04-26T01:46:38.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.020+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9ac005330 con 0x7fa9c01015f0 2024-04-26T01:46:38.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.021+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fa9b0029070 con 0x7fa9c01015f0 2024-04-26T01:46:38.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.021+0000 7fa9bd7fa700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa9a8039650 0x7fa9a803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:38.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.021+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa9b001e070 con 0x7fa9c01015f0 2024-04-26T01:46:38.019 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.022+0000 7fa9bf7fe700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa9a8039650 0x7fa9a803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:38.020 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.023+0000 7fa9bf7fe700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa9a8039650 0x7fa9a803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006fd0 tx=0x7fa9b4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:38.036 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.038+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9b0013070 con 0x7fa9c01015f0 2024-04-26T01:46:38.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:38 smithi071 systemd[1]: Starting Ceph mon.c for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:46:38.344 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.346+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa9ac0058f0 con 0x7fa9c01015f0 2024-04-26T01:46:38.345 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.347+0000 7fa9bd7fa700 1 -- 172.21.15.71:0/1609717789 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fa9b000d340 con 0x7fa9c01015f0 2024-04-26T01:46:38.345 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:46:38.345 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":1,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","modified":"2024-04-26T01:44:19.989980Z","created":"2024-04-26T01:44:19.989980Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa9a8039650 msgr2=0x7fa9a803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa9a8039650 0x7fa9a803bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4006fd0 tx=0x7fa9b4006e40 comp rx=0 tx=0).stop 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 msgr2=0x7fa9c0197e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c0197e80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa9b00056d0 tx=0x7fa9b00057b0 comp rx=0 tx=0).stop 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 shutdown_connections 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa9a8039650 0x7fa9a803bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 --2- 172.21.15.71:0/1609717789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9c01015f0 0x7fa9c0197e80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 >> 172.21.15.71:0/1609717789 conn(0x7fa9c0076ea0 msgr2=0x7fa9c0069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 shutdown_connections 2024-04-26T01:46:38.347 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:38.349+0000 7fa9c6799700 1 -- 172.21.15.71:0/1609717789 wait complete. 2024-04-26T01:46:38.348 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 1 2024-04-26T01:46:38.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:38 smithi037 ceph-mon[27906]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:38.610 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:38 smithi071 podman[32542]: 2024-04-26 01:46:38.392335049 +0000 UTC m=+0.019473310 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:46:38.610 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:38 smithi071 podman[32542]: 2024-04-26 01:46:38.492858606 +0000 UTC m=+0.119996855 container create b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 podman[32542]: 2024-04-26 01:46:39.123136325 +0000 UTC m=+0.750274574 container init b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 podman[32542]: 2024-04-26 01:46:39.126880658 +0000 UTC m=+0.754018907 container start b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pidfile_write: ignore empty --pid-file 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: load: jerasure load: lrc 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: RocksDB version: 7.9.2 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Git sha 0 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: DB SUMMARY 2024-04-26T01:46:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: DB Session ID: A5Z070NJKZQKD05W3YDJ 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: CURRENT file: CURRENT 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.error_if_exists: 0 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.create_if_missing: 0 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.paranoid_checks: 1 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.env: 0x55944dac8cc0 2024-04-26T01:46:39.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T01:46:39.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.info_log: 0x55944f20fbe0 2024-04-26T01:46:39.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T01:46:39.405 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.statistics: (nil) 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.use_fsync: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_log_file_size: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.allow_fallocate: 1 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.use_direct_reads: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T01:46:39.406 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.db_log_dir: 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.wal_dir: 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.write_buffer_manager: 0x55944f1a30e0 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T01:46:39.407 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.unordered_write: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.row_cache: None 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.wal_filter: None 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.two_write_queues: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T01:46:39.408 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.wal_compression: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.atomic_flush: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.log_readahead_size: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_background_jobs: 2 2024-04-26T01:46:39.409 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_background_compactions: -1 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_subcompactions: 1 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_open_files: -1 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T01:46:39.410 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_background_flushes: -1 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Compression algorithms supported: 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kZSTD supported: 0 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kXpressCompression supported: 0 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kZlibCompression supported: 1 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kSnappyCompression supported: 1 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kLZ4Compression supported: 1 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: kBZip2Compression supported: 0 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T01:46:39.411 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.merge_operator: 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_filter: None 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55944f20fdc0) 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: cache_index_and_filter_blocks: 1 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: pin_top_level_index_and_filter: 1 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_type: 0 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: data_block_index_type: 0 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_shortening: 1 2024-04-26T01:46:39.412 INFO:journalctl@ceph.mon.c.smithi071.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: checksum: 4 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: no_block_cache: 0 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache: 0x55944f1d51f0 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_name: BinnedLRUCache 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_options: 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: capacity : 536870912 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: num_shard_bits : 4 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: strict_capacity_limit : 0 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: high_pri_pool_ratio: 0.000 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_cache_compressed: (nil) 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: persistent_cache: (nil) 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_size: 4096 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_size_deviation: 10 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_restart_interval: 16 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: index_block_restart_interval: 1 2024-04-26T01:46:39.413 INFO:journalctl@ceph.mon.c.smithi071.stdout: metadata_block_size: 4096 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: partition_filters: 0 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: use_delta_encoding: 1 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: filter_policy: bloomfilter 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: whole_key_filtering: 1 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: verify_compression: 0 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: read_amp_bytes_per_bit: 0 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: format_version: 5 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: enable_index_compression: 1 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: block_align: 0 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: max_auto_readahead_size: 262144 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: prepopulate_block_cache: 0 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: initial_auto_readahead_size: 8192 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T01:46:39.414 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression: NoCompression 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.num_levels: 7 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T01:46:39.415 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T01:46:39.416 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T01:46:39.417 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T01:46:39.418 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.inplace_update_support: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.bloom_locality: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.max_successive_merges: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.ttl: 2592000 2024-04-26T01:46:39.419 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.enable_blob_files: false 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.min_blob_size: 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 28c24281-1904-4dd1-8f79-e5f039cb595b 2024-04-26T01:46:39.420 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095999157122, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095999157563, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714095999, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "28c24281-1904-4dd1-8f79-e5f039cb595b", "db_session_id": "A5Z070NJKZQKD05W3YDJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714095999157651, "job": 1, "event": "recovery_finished"} 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55944f2ea000 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: DB pointer 0x55944f2d4000 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** DB Stats ** 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:46:39.421 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** Compaction Stats [default] ** 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T01:46:39.422 INFO:journalctl@ceph.mon.c.smithi071.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Cumulative compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Interval compaction: 0.00 GB write, 0.58 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Block cache BinnedLRUCache@0x55944f1d51f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-04-26T01:46:39.423 INFO:journalctl@ceph.mon.c.smithi071.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: using public_addr v2:172.21.15.71:0/0 -> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: starting mon.c rank -1 at public addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] at bind addrs [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(???) e0 preinit fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout: e1 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout: legacy client fscid: -1 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout: 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout: No filesystems configured 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T01:46:39.424 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Deploying cephadm binary to smithi045 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.425 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Added host smithi045 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi045", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.426 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi071", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Deploying cephadm binary to smithi071 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.427 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Added host smithi071 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi071", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:39.428 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1253064705' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1253064705' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:39.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c;count:3 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: Deploying daemon mon.c on smithi071 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/1609717789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 ceph-mon[32578]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 bash[32542]: b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb 2024-04-26T01:46:39.430 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:39 smithi071 systemd[1]: Started Ceph mon.c for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:46:39.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:39 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/1609717789' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:46:40.025 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:46:40.025 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:46:40.259 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:46:44.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:46:44.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: mon.a calling monitor election 2024-04-26T01:46:44.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.535 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: mon.c calling monitor election 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: monmap epoch 2 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: last_changed 2024-04-26T01:46:39.191222+0000 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: min_mon_release 19 (squid) 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: election_strategy: 1 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:44.536 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: fsmap 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: mgrmap e13: x(active, since 61s) 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:44.537 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:44.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:46:44.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: mon.a calling monitor election 2024-04-26T01:46:44.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: mon.c calling monitor election 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: monmap epoch 2 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: last_changed 2024-04-26T01:46:39.191222+0000 2024-04-26T01:46:44.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: min_mon_release 19 (squid) 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: election_strategy: 1 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: fsmap 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: mgrmap e13: x(active, since 61s) 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: overall HEALTH_OK 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:44.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:44.711 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:44.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 -- 172.21.15.71:0/55618280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 msgr2=0x7ff7e4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:44.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 --2- 172.21.15.71:0/55618280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7e4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc079c80 tx=0x7ff7ec00feb0 comp rx=0 tx=0).stop 2024-04-26T01:46:44.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 -- 172.21.15.71:0/55618280 shutdown_connections 2024-04-26T01:46:44.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 --2- 172.21.15.71:0/55618280 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7e4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:44.774 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 -- 172.21.15.71:0/55618280 >> 172.21.15.71:0/55618280 conn(0x7ff7fc077af0 msgr2=0x7ff7fc077f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:44.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 -- 172.21.15.71:0/55618280 shutdown_connections 2024-04-26T01:46:44.775 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.776+0000 7ff801cd9700 1 -- 172.21.15.71:0/55618280 wait complete. 2024-04-26T01:46:44.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.777+0000 7ff801cd9700 1 Processor -- start 2024-04-26T01:46:44.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff801cd9700 1 -- start start 2024-04-26T01:46:44.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff801cd9700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7fc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:44.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff801cd9700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:44.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff801cd9700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc10a4b0 con 0x7ff7e40035c0 2024-04-26T01:46:44.776 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff801cd9700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff7fc10a330 con 0x7ff7fc1984b0 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48046/0 (socket says 172.21.15.71:48046) 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 -- 172.21.15.71:0/1194998166 learned_addr learned my addr 172.21.15.71:0/1194998166 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 -- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 msgr2=0x7ff7fc19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 -- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 msgr2=0x7ff7fc19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7faffd700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:46:44.777 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.778+0000 7ff7fb7fe700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7fc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.779+0000 7ff7fb7fe700 1 -- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 msgr2=0x7ff7fc19c840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:46:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.779+0000 7ff7fb7fe700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.779+0000 7ff7fb7fe700 1 -- 172.21.15.71:0/1194998166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7fc198af0 con 0x7ff7e40035c0 2024-04-26T01:46:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.780+0000 7ff7fb7fe700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7fc197f70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000c00 tx=0x7ff7ec0046f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.780+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff7ec013b00 con 0x7ff7e40035c0 2024-04-26T01:46:44.778 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.780+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7ec0044e0 con 0x7ff7e40035c0 2024-04-26T01:46:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.780+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7fc19cd80 con 0x7ff7e40035c0 2024-04-26T01:46:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.780+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7ff7ec011470 con 0x7ff7e40035c0 2024-04-26T01:46:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.780+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7fc19d360 con 0x7ff7e40035c0 2024-04-26T01:46:44.779 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.781+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7fc06a9a0 con 0x7ff7e40035c0 2024-04-26T01:46:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.782+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff7ec023070 con 0x7ff7e40035c0 2024-04-26T01:46:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.782+0000 7ff7f8ff9700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff7e8039440 0x7ff7e803b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.782+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff7ec020050 con 0x7ff7e40035c0 2024-04-26T01:46:44.780 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.782+0000 7ff7faffd700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff7e8039440 0x7ff7e803b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:44.781 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.784+0000 7ff7faffd700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff7e8039440 0x7ff7e803b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009180 tx=0x7ff7f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:44.796 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:44.798+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff7ec028020 con 0x7ff7e40035c0 2024-04-26T01:46:45.084 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.074+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff7fc199360 con 0x7ff7e40035c0 2024-04-26T01:46:45.086 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.075+0000 7ff7f8ff9700 1 -- 172.21.15.71:0/1194998166 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1011 (secure 0 0 0) 0x7ff7fc199360 con 0x7ff7e40035c0 2024-04-26T01:46:45.086 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:46:45.087 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":2,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","modified":"2024-04-26T01:46:39.191222Z","created":"2024-04-26T01:44:19.989980Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:46:45.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff7e8039440 msgr2=0x7ff7e803b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:45.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff7e8039440 0x7ff7e803b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009180 tx=0x7ff7f0008040 comp rx=0 tx=0).stop 2024-04-26T01:46:45.088 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 msgr2=0x7ff7fc197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7fc197f70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec000c00 tx=0x7ff7ec0046f0 comp rx=0 tx=0).stop 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 shutdown_connections 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7e40035c0 0x7ff7fc197f70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff7e8039440 0x7ff7e803b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 --2- 172.21.15.71:0/1194998166 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff7fc1984b0 0x7ff7fc19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.077+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 >> 172.21.15.71:0/1194998166 conn(0x7ff7fc077af0 msgr2=0x7ff7fc0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:45.089 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.078+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 shutdown_connections 2024-04-26T01:46:45.090 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:45.078+0000 7ff801cd9700 1 -- 172.21.15.71:0/1194998166 wait complete. 2024-04-26T01:46:45.090 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 2 2024-04-26T01:46:45.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:45 smithi071 ceph-mon[32578]: Deploying daemon mon.b on smithi045 2024-04-26T01:46:45.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:45 smithi071 ceph-mon[32578]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:45.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:45 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/1194998166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:46:45.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:45 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:45.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:45 smithi037 ceph-mon[27906]: Deploying daemon mon.b on smithi045 2024-04-26T01:46:45.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:45 smithi037 ceph-mon[27906]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:45.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:45 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/1194998166' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T01:46:45.435 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:45 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:46.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:46:46 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:46:46.190+0000 7f6481a00700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T01:46:46.653 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T01:46:46.654 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T01:46:46.895 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:46:47.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:47 smithi071 ceph-mon[32578]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:47.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:47 smithi037 ceph-mon[27906]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:48.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:48 smithi045 systemd[1]: Started Ceph mon.b for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:46:53.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: mon.c calling monitor election 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: mon.a calling monitor election 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: monmap epoch 3 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: last_changed 2024-04-26T01:46:48.169099+0000 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: min_mon_release 19 (squid) 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: election_strategy: 1 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: fsmap 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: mgrmap e13: x(active, since 70s) 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T01:46:53.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: mon.c calling monitor election 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: mon.a calling monitor election 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: monmap epoch 3 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: last_changed 2024-04-26T01:46:48.169099+0000 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: min_mon_release 19 (squid) 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: election_strategy: 1 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: fsmap 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: mgrmap e13: x(active, since 70s) 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: overall HEALTH_OK 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:53.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:54.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 -- 172.21.15.71:0/4123050881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4003a40 msgr2=0x7f43d40062a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 --2- 172.21.15.71:0/4123050881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4003a40 0x7f43d40062a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f43ec102510 tx=0x7f43e000fe90 comp rx=0 tx=0).stop 2024-04-26T01:46:54.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 -- 172.21.15.71:0/4123050881 shutdown_connections 2024-04-26T01:46:54.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 --2- 172.21.15.71:0/4123050881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43d4003a40 0x7f43d40062a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.370 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 -- 172.21.15.71:0/4123050881 >> 172.21.15.71:0/4123050881 conn(0x7f43ec0fcf80 msgr2=0x7f43ec0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:54.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 -- 172.21.15.71:0/4123050881 shutdown_connections 2024-04-26T01:46:54.371 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.372+0000 7f43f11ab700 1 -- 172.21.15.71:0/4123050881 wait complete. 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.373+0000 7f43f11ab700 1 Processor -- start 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.373+0000 7f43f11ab700 1 -- start start 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43f11ab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 0x7f43ec1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43f11ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43ec197630 0x7f43ec19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43f11ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 0x7f43ec19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43f11ab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec109340 con 0x7f43ec197630 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43f11ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1091c0 con 0x7f43ec19bf00 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43f11ab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f43ec1094c0 con 0x7f43d4003a40 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 0x7f43ec1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 0x7f43ec1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38414/0 (socket says 172.21.15.71:38414) 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 -- 172.21.15.71:0/2785837635 learned_addr learned my addr 172.21.15.71:0/2785837635 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43eb59e700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 0x7f43ec19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.373 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ea59c700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43ec197630 0x7f43ec19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43eb59e700 1 -- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 msgr2=0x7f43ec19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43eb59e700 1 -- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 msgr2=0x7f43ec19c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43eb59e700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 0x7f43ec19c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43eb59e700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 0x7f43ec19c3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 -- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 msgr2=0x7f43ec19c3b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 0x7f43ec19c3b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 -- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43ec197630 msgr2=0x7f43ec19b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43ec197630 0x7f43ec19b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 -- 172.21.15.71:0/2785837635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43ec1a3c70 con 0x7f43d4003a40 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ea59c700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43ec197630 0x7f43ec19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.374+0000 7f43ead9d700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 0x7f43ec1970f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43e0015040 tx=0x7f43e0012cb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:54.374 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.375+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0011de0 con 0x7f43d4003a40 2024-04-26T01:46:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.375+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43ec1a3e40 con 0x7f43d4003a40 2024-04-26T01:46:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.375+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43e002a070 con 0x7f43d4003a40 2024-04-26T01:46:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.375+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43e0003680 con 0x7f43d4003a40 2024-04-26T01:46:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.375+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43ec1a42f0 con 0x7f43d4003a40 2024-04-26T01:46:54.375 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.376+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f43e0003820 con 0x7f43d4003a40 2024-04-26T01:46:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.376+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43ec0640a0 con 0x7f43d4003a40 2024-04-26T01:46:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.376+0000 7f43d3fff700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f43d8039ae0 0x7f43d803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.376+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f43e004f770 con 0x7f43d4003a40 2024-04-26T01:46:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.376+0000 7f43ea59c700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f43d8039ae0 0x7f43d803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:54.376 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.377+0000 7f43ea59c700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f43d8039ae0 0x7f43d803bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0079b0 tx=0x7f43dc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:54.379 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.381+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43e002d2e0 con 0x7f43d4003a40 2024-04-26T01:46:54.573 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:54.574 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:54.667 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.669+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f43ec1a4550 con 0x7f43d4003a40 2024-04-26T01:46:54.668 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.670+0000 7f43d3fff700 1 -- 172.21.15.71:0/2785837635 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1275 (secure 0 0 0) 0x7f43e000cd70 con 0x7f43d4003a40 2024-04-26T01:46:54.668 INFO:teuthology.orchestra.run.smithi071.stdout: 2024-04-26T01:46:54.668 INFO:teuthology.orchestra.run.smithi071.stdout:{"epoch":3,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","modified":"2024-04-26T01:46:48.169099Z","created":"2024-04-26T01:44:19.989980Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:3300","nonce":0},{"type":"v1","addr":"172.21.15.71:6789","nonce":0}]},"addr":"172.21.15.71:6789/0","public_addr":"172.21.15.71:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:3300","nonce":0},{"type":"v1","addr":"172.21.15.45:6789","nonce":0}]},"addr":"172.21.15.45:6789/0","public_addr":"172.21.15.45:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f43d8039ae0 msgr2=0x7f43d803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f43d8039ae0 0x7f43d803bfa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f43dc0079b0 tx=0x7f43dc008040 comp rx=0 tx=0).stop 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 msgr2=0x7f43ec1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 0x7f43ec1970f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f43e0015040 tx=0x7f43e0012cb0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 shutdown_connections 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f43d8039ae0 0x7f43d803bfa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f43ec19bf00 0x7f43ec19c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f43d4003a40 0x7f43ec1970f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.670 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 --2- 172.21.15.71:0/2785837635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43ec197630 0x7f43ec19b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:54.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 >> 172.21.15.71:0/2785837635 conn(0x7f43ec0fcf80 msgr2=0x7f43ec102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:54.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 shutdown_connections 2024-04-26T01:46:54.671 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:46:54.672+0000 7f43f11ab700 1 -- 172.21.15.71:0/2785837635 wait complete. 2024-04-26T01:46:54.671 INFO:teuthology.orchestra.run.smithi071.stderr:dumped monmap epoch 3 2024-04-26T01:46:54.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:54.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.c calling monitor election 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.a calling monitor election 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: monmap epoch 3 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: last_changed 2024-04-26T01:46:48.169099+0000 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: min_mon_release 19 (squid) 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: election_strategy: 1 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: fsmap 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mgrmap e13: x(active, since 70s) 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: overall HEALTH_OK 2024-04-26T01:46:55.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:55.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:55.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:55.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:55.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:46:55.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:55.528 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T01:46:55.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T01:46:55.968 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:46:56.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: mon.b calling monitor election 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: mon.b calling monitor election 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: mon.c calling monitor election 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: mon.a calling monitor election 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: monmap epoch 3 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: last_changed 2024-04-26T01:46:48.169099+0000 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: min_mon_release 19 (squid) 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: election_strategy: 1 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: fsmap 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: mgrmap e13: x(active, since 72s) 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: mon.b calling monitor election 2024-04-26T01:46:56.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:56.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: mon.b calling monitor election 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: mon.c calling monitor election 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: mon.a calling monitor election 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: monmap epoch 3 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: last_changed 2024-04-26T01:46:48.169099+0000 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: min_mon_release 19 (squid) 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: election_strategy: 1 2024-04-26T01:46:56.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: fsmap 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: mgrmap e13: x(active, since 72s) 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: overall HEALTH_OK 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.b calling monitor election 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.b calling monitor election 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.c calling monitor election 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.a calling monitor election 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: monmap epoch 3 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: last_changed 2024-04-26T01:46:48.169099+0000 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: created 2024-04-26T01:44:19.989980+0000 2024-04-26T01:46:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: min_mon_release 19 (squid) 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: election_strategy: 1 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: 1: [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] mon.c 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: 2: [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] mon.b 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: fsmap 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: mgrmap e13: x(active, since 72s) 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: overall HEALTH_OK 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:46:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:57.170 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.170 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:46:57.171 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:57.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:46:57.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:46:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:57.443 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:46:57 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:46:57.169+0000 7f6481a00700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T01:46:58.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:57 smithi037 ceph-mon[27906]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:58.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:57 smithi037 ceph-mon[27906]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:46:58.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:57 smithi037 ceph-mon[27906]: Reconfiguring daemon mon.a on smithi037 2024-04-26T01:46:58.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:58.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:57 smithi071 ceph-mon[32578]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:58.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:57 smithi071 ceph-mon[32578]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:46:58.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:57 smithi071 ceph-mon[32578]: Reconfiguring daemon mon.a on smithi037 2024-04-26T01:46:58.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:58.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:57 smithi045 ceph-mon[32520]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:46:58.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:57 smithi045 ceph-mon[32520]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T01:46:58.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:57 smithi045 ceph-mon[32520]: Reconfiguring daemon mon.a on smithi037 2024-04-26T01:46:58.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:46:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 -- 172.21.15.37:0/2710798616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c074e70 msgr2=0x7f9d7c075250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2710798616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c074e70 0x7f9d7c075250 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009910 tx=0x7f9d6c009c20 comp rx=0 tx=0).stop 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 -- 172.21.15.37:0/2710798616 shutdown_connections 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2710798616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c074e70 0x7f9d7c075250 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 -- 172.21.15.37:0/2710798616 >> 172.21.15.37:0/2710798616 conn(0x7f9d7c0fdf80 msgr2=0x7f9d7c1003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 -- 172.21.15.37:0/2710798616 shutdown_connections 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.237+0000 7f9d82c00700 1 -- 172.21.15.37:0/2710798616 wait complete. 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.238+0000 7f9d82c00700 1 Processor -- start 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 -- start start 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7c074e70 0x7f9d7c19a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 0x7f9d7c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d7c19f190 0x7f9d7c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c10ebb0 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c10ea30 con 0x7f9d7c074e70 2024-04-26T01:46:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d82c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7c10ed30 con 0x7f9d7c19f190 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d73400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 0x7f9d7c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d7be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7c074e70 0x7f9d7c19a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d73400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 0x7f9d7c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40866/0 (socket says 172.21.15.37:40866) 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.239+0000 7f9d73400700 1 -- 172.21.15.37:0/2357252181 learned_addr learned my addr 172.21.15.37:0/2357252181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d80c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d7c19f190 0x7f9d7c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d73400700 1 -- 172.21.15.37:0/2357252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d7c19f190 msgr2=0x7f9d7c19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d73400700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d7c19f190 0x7f9d7c19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d73400700 1 -- 172.21.15.37:0/2357252181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7c074e70 msgr2=0x7f9d7c19a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d73400700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7c074e70 0x7f9d7c19a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d73400700 1 -- 172.21.15.37:0/2357252181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7c1a6f00 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.240+0000 7f9d73400700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 0x7f9d7c19ec50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400b780 tx=0x7f9d6400ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.241+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64016070 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.241+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d640036a0 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.241+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7c1a7130 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.241+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d640119c0 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.241+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d7c1a76c0 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.242+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d7c0640a0 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.242+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9d64019070 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.242+0000 7f9d79600700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9d60039ae0 0x7f9d6003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.243+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9d6404b350 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.243+0000 7f9d7be00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9d60039ae0 0x7f9d6003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.244+0000 7f9d7be00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9d60039ae0 0x7f9d6003bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006010 tx=0x7f9d6c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.249+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d6401ad70 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.481+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f9d7c1a7980 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.482+0000 7f9d79600700 1 -- 172.21.15.37:0/2357252181 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+271 (secure 0 0 0) 0x7f9d64014120 con 0x7f9d7c19a8c0 2024-04-26T01:46:58.511 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9d60039ae0 msgr2=0x7f9d6003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9d60039ae0 0x7f9d6003bfa0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006010 tx=0x7f9d6c01a040 comp rx=0 tx=0).stop 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 msgr2=0x7f9d7c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 0x7f9d7c19ec50 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400b780 tx=0x7f9d6400ba90 comp rx=0 tx=0).stop 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 shutdown_connections 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9d60039ae0 0x7f9d6003bfa0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d7c19f190 0x7f9d7c19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d7c074e70 0x7f9d7c19a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 --2- 172.21.15.37:0/2357252181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d7c19a8c0 0x7f9d7c19ec50 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:46:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 >> 172.21.15.37:0/2357252181 conn(0x7f9d7c0fdf80 msgr2=0x7f9d7c108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:46:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.485+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 shutdown_connections 2024-04-26T01:46:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:46:58.486+0000 7f9d82c00700 1 -- 172.21.15.37:0/2357252181 wait complete. 2024-04-26T01:46:59.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:58 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2357252181' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:59.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:58 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2357252181' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:59.341 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T01:46:59.341 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:46:59.341 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:46:59.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:58 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2357252181' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:46:59.420 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:46:59.420 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:59.540 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:46:59.540 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:46:59.573 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:46:59.573 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:59.641 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:46:59.642 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T01:46:59.673 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:46:59.673 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:46:59.741 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-04-26T01:46:59.741 INFO:tasks.cephadm:Adding mgr.y on smithi045 2024-04-26T01:46:59.741 INFO:tasks.cephadm:Adding mgr.z on smithi071 2024-04-26T01:46:59.741 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi045=y;smithi071=z' 2024-04-26T01:47:00.017 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:47:00.138 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:59 smithi071 ceph-mon[32578]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:00.139 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:59 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:00.139 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:59 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:00.139 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:59 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:47:00.139 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:59 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:47:00.139 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:46:59 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:00.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:59 smithi037 ceph-mon[27906]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:00.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:59 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:00.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:59 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:00.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:59 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:47:00.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:59 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:47:00.201 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:46:59 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:00.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:59 smithi045 ceph-mon[32520]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:00.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:59 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:00.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:59 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:00.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:59 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:47:00.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:59 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:47:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:46:59 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:01.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:00 smithi045 ceph-mon[32520]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:47:01.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:00 smithi045 ceph-mon[32520]: Reconfiguring daemon mon.b on smithi045 2024-04-26T01:47:01.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:00 smithi037 ceph-mon[27906]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:47:01.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:00 smithi037 ceph-mon[27906]: Reconfiguring daemon mon.b on smithi045 2024-04-26T01:47:01.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:00 smithi071 ceph-mon[32578]: Reconfiguring mon.b (monmap changed)... 2024-04-26T01:47:01.222 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:00 smithi071 ceph-mon[32578]: Reconfiguring daemon mon.b on smithi045 2024-04-26T01:47:01.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.441+0000 7f9ccbfff700 1 -- 172.21.15.71:0/4122911653 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0005660 con 0x7f9cd41014e0 2024-04-26T01:47:01.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.441+0000 7f9cdb397700 1 -- 172.21.15.71:0/4122911653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 msgr2=0x7f9cd4101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:01.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.441+0000 7f9cdb397700 1 --2- 172.21.15.71:0/4122911653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 0x7f9cd4101900 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0009910 tx=0x7f9cd0009c20 comp rx=0 tx=0).stop 2024-04-26T01:47:01.439 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.441+0000 7f9cdb397700 1 -- 172.21.15.71:0/4122911653 shutdown_connections 2024-04-26T01:47:01.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.441+0000 7f9cdb397700 1 --2- 172.21.15.71:0/4122911653 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 0x7f9cd4101900 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.441+0000 7f9cdb397700 1 -- 172.21.15.71:0/4122911653 >> 172.21.15.71:0/4122911653 conn(0x7f9cd40fca80 msgr2=0x7f9cd40feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:01.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 -- 172.21.15.71:0/4122911653 shutdown_connections 2024-04-26T01:47:01.440 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 -- 172.21.15.71:0/4122911653 wait complete. 2024-04-26T01:47:01.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 Processor -- start 2024-04-26T01:47:01.442 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 -- start start 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 0x7f9cd4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cd41982f0 0x7f9cd419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 0x7f9cd419f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd4107420 con 0x7f9cd41014e0 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41072a0 con 0x7f9cd41982f0 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cdb397700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd41075a0 con 0x7f9cd4198770 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.442+0000 7f9cd9934700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 0x7f9cd419f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 0x7f9cd419f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:38426/0 (socket says 172.21.15.71:38426) 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 -- 172.21.15.71:0/3511272102 learned_addr learned my addr 172.21.15.71:0/3511272102 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd8932700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cd41982f0 0x7f9cd419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 -- 172.21.15.71:0/3511272102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cd41982f0 msgr2=0x7f9cd419d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cd41982f0 0x7f9cd419d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 -- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 msgr2=0x7f9cd4197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:01.443 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 0x7f9cd4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 -- 172.21.15.71:0/3511272102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd41a01d0 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd8932700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cd41982f0 0x7f9cd419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cd9934700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 0x7f9cd419f8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00ba50 tx=0x7f9ccc00be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc019070 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ccc014440 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd41a04c0 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc007590 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.443+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41a0a30 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.444+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9ccc01c070 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.444+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd40640a0 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.444+0000 7f9cca7fc700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9cc0039ae0 0x7f9cc003bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.444+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9ccc04bf30 con 0x7f9cd4198770 2024-04-26T01:47:01.444 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.445+0000 7f9cd9133700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9cc0039ae0 0x7f9cc003bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:01.445 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.446+0000 7f9cd9133700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9cc0039ae0 0x7f9cc003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005e00 tx=0x7f9cd0005260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:01.447 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.450+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ccc01b2c0 con 0x7f9cd4198770 2024-04-26T01:47:01.664 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.667+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi045=y;smithi071=z", "target": ["mon-mgr", ""]}) -- 0x7f9cd4062f40 con 0x7f9cc0039ae0 2024-04-26T01:47:01.678 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.680+0000 7f9cca7fc700 1 -- 172.21.15.71:0/3511272102 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9cd4062f40 con 0x7f9cc0039ae0 2024-04-26T01:47:01.678 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mgr update... 2024-04-26T01:47:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9cc0039ae0 msgr2=0x7f9cc003bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9cc0039ae0 0x7f9cc003bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9cd0005e00 tx=0x7f9cd0005260 comp rx=0 tx=0).stop 2024-04-26T01:47:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 msgr2=0x7f9cd419f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 0x7f9cd419f8d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc00ba50 tx=0x7f9ccc00be10 comp rx=0 tx=0).stop 2024-04-26T01:47:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 shutdown_connections 2024-04-26T01:47:01.680 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9cd41014e0 0x7f9cd4197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9cc0039ae0 0x7f9cc003bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9cd4198770 0x7f9cd419f8d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 --2- 172.21.15.71:0/3511272102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9cd41982f0 0x7f9cd419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 >> 172.21.15.71:0/3511272102 conn(0x7f9cd40fca80 msgr2=0x7f9cd406b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 shutdown_connections 2024-04-26T01:47:01.681 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:47:01.682+0000 7f9cdb397700 1 -- 172.21.15.71:0/3511272102 wait complete. 2024-04-26T01:47:02.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:02.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:47:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:47:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:02.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:47:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:47:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.262 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:02.262 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.263 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.263 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T01:47:02.263 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T01:47:02.263 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:02.263 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:02.288 DEBUG:teuthology.orchestra.run.smithi045:mgr.y> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T01:47:02.291 DEBUG:teuthology.orchestra.run.smithi071:mgr.z> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T01:47:02.293 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T01:47:02.294 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:47:02.294 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:47:02.356 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:47:02.356 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 01:46:00.383304828 +0000 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 01:40:37.005348762 +0000 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 01:40:37.005348762 +0000 2024-04-26T01:47:02.415 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 01:40:37.005348762 +0000 2024-04-26T01:47:02.416 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:47:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T01:47:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T01:47:02.504 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000399845 s, 1.3 MB/s 2024-04-26T01:47:02.508 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:47:02.565 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-04-26T01:47:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:47:02.618 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:02.618 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-26T01:47:02.618 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:02.618 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:02.618 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 01:46:00.465303717 +0000 2024-04-26T01:47:02.619 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 01:40:37.631340538 +0000 2024-04-26T01:47:02.619 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 01:40:37.631340538 +0000 2024-04-26T01:47:02.619 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 01:40:37.631340538 +0000 2024-04-26T01:47:02.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:47:02.701 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T01:47:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T01:47:02.702 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000367674 s, 1.4 MB/s 2024-04-26T01:47:02.705 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:47:02.767 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 01:46:00.534302782 +0000 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 01:40:38.301331737 +0000 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 01:40:38.301331737 +0000 2024-04-26T01:47:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 01:40:38.301331737 +0000 2024-04-26T01:47:02.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:47:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T01:47:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T01:47:02.905 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000366169 s, 1.4 MB/s 2024-04-26T01:47:02.909 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:47:02.967 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-04-26T01:47:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:47:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-26T01:47:03.019 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 01:46:00.612301726 +0000 2024-04-26T01:47:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 01:40:38.959323093 +0000 2024-04-26T01:47:03.020 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 01:40:38.959323093 +0000 2024-04-26T01:47:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 01:40:38.959323093 +0000 2024-04-26T01:47:03.020 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:47:03.108 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-04-26T01:47:03.108 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-04-26T01:47:03.108 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00043111 s, 1.2 MB/s 2024-04-26T01:47:03.112 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:47:03.172 DEBUG:teuthology.orchestra.run.smithi045:> set -ex 2024-04-26T01:47:03.172 DEBUG:teuthology.orchestra.run.smithi045:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:47:03.192 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:47:03.192 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_1 2024-04-26T01:47:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:02 smithi037 ceph-mon[27906]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:47:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:02 smithi037 ceph-mon[27906]: Reconfiguring daemon mon.c on smithi071 2024-04-26T01:47:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:02 smithi037 ceph-mon[27906]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi045=y;smithi071=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:02 smithi037 ceph-mon[27906]: Saving service mgr spec with placement smithi037=x;smithi045=y;smithi071=z;count:3 2024-04-26T01:47:03.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:02 smithi071 ceph-mon[32578]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:47:03.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:02 smithi071 ceph-mon[32578]: Reconfiguring daemon mon.c on smithi071 2024-04-26T01:47:03.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:02 smithi071 ceph-mon[32578]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi045=y;smithi071=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:03.210 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:02 smithi071 ceph-mon[32578]: Saving service mgr spec with placement smithi037=x;smithi045=y;smithi071=z;count:3 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-26 01:46:47.611699977 +0000 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-26 01:39:25.286717150 +0000 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-26 01:39:25.286717150 +0000 2024-04-26T01:47:03.251 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-26 01:39:25.286717150 +0000 2024-04-26T01:47:03.251 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:47:03.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:02 smithi045 ceph-mon[32520]: Reconfiguring mon.c (monmap changed)... 2024-04-26T01:47:03.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:02 smithi045 ceph-mon[32520]: Reconfiguring daemon mon.c on smithi071 2024-04-26T01:47:03.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:02 smithi045 ceph-mon[32520]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi045=y;smithi071=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:03.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:02 smithi045 ceph-mon[32520]: Saving service mgr spec with placement smithi037=x;smithi045=y;smithi071=z;count:3 2024-04-26T01:47:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-26T01:47:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-26T01:47:03.280 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000126405 s, 4.1 MB/s 2024-04-26T01:47:03.282 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:47:03.342 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_2 2024-04-26T01:47:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:47:03.399 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:03.399 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T01:47:03.399 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:03.400 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:03.400 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-26 01:46:47.611699977 +0000 2024-04-26T01:47:03.400 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-26 01:39:25.645711324 +0000 2024-04-26T01:47:03.400 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-26 01:39:25.645711324 +0000 2024-04-26T01:47:03.400 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-26 01:39:25.645711324 +0000 2024-04-26T01:47:03.400 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:47:03.464 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-26T01:47:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-26T01:47:03.465 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000117953 s, 4.3 MB/s 2024-04-26T01:47:03.466 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:47:03.523 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_3 2024-04-26T01:47:03.579 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-26 01:46:47.611699977 +0000 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-26 01:39:25.988705758 +0000 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-26 01:39:25.988705758 +0000 2024-04-26T01:47:03.580 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-26 01:39:25.988705758 +0000 2024-04-26T01:47:03.580 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:47:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-26T01:47:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-26T01:47:03.645 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.00011582 s, 4.4 MB/s 2024-04-26T01:47:03.646 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:47:03.703 DEBUG:teuthology.orchestra.run.smithi045:> stat /dev/vg_nvme/lv_4 2024-04-26T01:47:03.760 INFO:teuthology.orchestra.run.smithi045.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Access: 2024-04-26 01:46:47.612699962 +0000 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Modify: 2024-04-26 01:39:26.317700419 +0000 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout:Change: 2024-04-26 01:39:26.317700419 +0000 2024-04-26T01:47:03.761 INFO:teuthology.orchestra.run.smithi045.stdout: Birth: 2024-04-26 01:39:26.317700419 +0000 2024-04-26T01:47:03.761 DEBUG:teuthology.orchestra.run.smithi045:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:47:03.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records in 2024-04-26T01:47:03.826 INFO:teuthology.orchestra.run.smithi045.stderr:1+0 records out 2024-04-26T01:47:03.826 INFO:teuthology.orchestra.run.smithi045.stderr:512 bytes copied, 0.000119128 s, 4.3 MB/s 2024-04-26T01:47:03.827 DEBUG:teuthology.orchestra.run.smithi045:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:47:03.885 DEBUG:teuthology.orchestra.run.smithi071:> set -ex 2024-04-26T01:47:03.885 DEBUG:teuthology.orchestra.run.smithi071:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T01:47:03.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T01:47:03.900 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_1 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-26 01:46:38.185713162 +0000 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-26 01:39:22.976724245 +0000 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-26 01:39:22.976724245 +0000 2024-04-26T01:47:03.955 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-26 01:39:22.976724245 +0000 2024-04-26T01:47:03.955 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T01:47:04.018 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-26T01:47:04.018 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-26T01:47:04.018 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000209288 s, 2.4 MB/s 2024-04-26T01:47:04.019 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T01:47:04.075 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_2 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-26 01:46:38.186713145 +0000 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-26 01:39:23.308718410 +0000 2024-04-26T01:47:04.130 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-26 01:39:23.308718410 +0000 2024-04-26T01:47:04.131 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-26 01:39:23.308718410 +0000 2024-04-26T01:47:04.131 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.190 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:04.191 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:47:04.191 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:04.191 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:04.191 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-26T01:47:04.192 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-26T01:47:04.192 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000159582 s, 3.2 MB/s 2024-04-26T01:47:04.193 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T01:47:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:04.248 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_3 2024-04-26T01:47:04.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:04.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-26 01:46:38.186713145 +0000 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-26 01:39:23.624712857 +0000 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-26 01:39:23.624712857 +0000 2024-04-26T01:47:04.303 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-26 01:39:23.624712857 +0000 2024-04-26T01:47:04.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T01:47:04.366 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-26T01:47:04.367 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-26T01:47:04.367 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.000180854 s, 2.8 MB/s 2024-04-26T01:47:04.368 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T01:47:04.423 DEBUG:teuthology.orchestra.run.smithi071:> stat /dev/vg_nvme/lv_4 2024-04-26T01:47:04.478 INFO:teuthology.orchestra.run.smithi071.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T01:47:04.478 INFO:teuthology.orchestra.run.smithi071.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T01:47:04.478 INFO:teuthology.orchestra.run.smithi071.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T01:47:04.478 INFO:teuthology.orchestra.run.smithi071.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:47:04.478 INFO:teuthology.orchestra.run.smithi071.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T01:47:04.478 INFO:teuthology.orchestra.run.smithi071.stdout:Access: 2024-04-26 01:46:38.186713145 +0000 2024-04-26T01:47:04.479 INFO:teuthology.orchestra.run.smithi071.stdout:Modify: 2024-04-26 01:39:23.960706951 +0000 2024-04-26T01:47:04.479 INFO:teuthology.orchestra.run.smithi071.stdout:Change: 2024-04-26 01:39:23.960706951 +0000 2024-04-26T01:47:04.479 INFO:teuthology.orchestra.run.smithi071.stdout: Birth: 2024-04-26 01:39:23.960706951 +0000 2024-04-26T01:47:04.479 DEBUG:teuthology.orchestra.run.smithi071:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T01:47:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records in 2024-04-26T01:47:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:1+0 records out 2024-04-26T01:47:04.540 INFO:teuthology.orchestra.run.smithi071.stderr:512 bytes copied, 0.00017494 s, 2.9 MB/s 2024-04-26T01:47:04.541 DEBUG:teuthology.orchestra.run.smithi071:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T01:47:04.597 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-04-26T01:47:04.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:47:05.058 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:05.198 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:04 smithi071 ceph-mon[32578]: Deploying daemon mgr.z on smithi071 2024-04-26T01:47:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:04 smithi037 ceph-mon[27906]: Deploying daemon mgr.z on smithi071 2024-04-26T01:47:05.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:04 smithi045 ceph-mon[32520]: Deploying daemon mgr.z on smithi071 2024-04-26T01:47:05.585 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:05 smithi071 systemd[1]: Starting Ceph mgr.z for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:47:05.901 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:05 smithi071 podman[34413]: 2024-04-26 01:47:05.587906845 +0000 UTC m=+0.113349315 container create 3f8112940c8a74dc3c42bab34ad5dca350d05bc355e86f38638cc9cae5f2bc31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T01:47:05.901 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:05 smithi071 podman[34413]: 2024-04-26 01:47:05.491769262 +0000 UTC m=+0.017211756 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:47:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:05 smithi037 ceph-mon[27906]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:05 smithi071 ceph-mon[32578]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:05 smithi071 podman[34413]: 2024-04-26 01:47:05.906621724 +0000 UTC m=+0.432064196 container init 3f8112940c8a74dc3c42bab34ad5dca350d05bc355e86f38638cc9cae5f2bc31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:05 smithi071 podman[34413]: 2024-04-26 01:47:05.909268975 +0000 UTC m=+0.434711455 container start 3f8112940c8a74dc3c42bab34ad5dca350d05bc355e86f38638cc9cae5f2bc31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:05 smithi071 ceph-mgr[34432]: -- 172.21.15.71:0/500387313 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56258a8685a0 con 0x56258a83cc00 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 bash[34413]: 3f8112940c8a74dc3c42bab34ad5dca350d05bc355e86f38638cc9cae5f2bc31 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 systemd[1]: Started Ceph mgr.z for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:06.059+0000 7f8c0bb12200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:47:06.244 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:06.246+0000 7f8c0bb12200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:47:06.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:05 smithi045 ceph-mon[32520]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:06.650 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:06.347+0000 7f8c0bb12200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:47:06.901 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:06.652+0000 7f8c0bb12200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:47:06.901 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:06.736+0000 7f8c0bb12200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:47:07.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:07.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:47:07.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:07.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:07.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-mon[32578]: Deploying daemon mgr.y on smithi045 2024-04-26T01:47:07.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:07.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:47:07.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:07.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:07.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:07 smithi037 ceph-mon[27906]: Deploying daemon mgr.y on smithi045 2024-04-26T01:47:07.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:07 smithi045 ceph-mon[32520]: Deploying daemon mgr.y on smithi045 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0369259 s, 284 MB/s 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 2024-04-26T01:47:07.884 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T01:47:08.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:08 smithi045 ceph-mon[32520]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:08.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:08 smithi071 ceph-mon[32578]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:08.265 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:07.930+0000 7f8c0bb12200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:47:08.265 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:08.097+0000 7f8c0bb12200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:47:08.266 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:08.267+0000 7f8c0bb12200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:47:08.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:08 smithi037 ceph-mon[27906]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:08.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-04-26T01:47:08.839 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:08.550+0000 7f8c0bb12200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:47:08.839 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:08.640+0000 7f8c0bb12200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:47:08.840 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:08.842+0000 7f8c0bb12200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:47:08.997 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:09.000+0000 7f22f8381200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:47:09.151 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:09.032+0000 7f8c0bb12200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:47:09.204 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:47:09.253 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:09.107+0000 7f22f8381200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:47:09.753 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:09.438+0000 7f22f8381200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:47:09.753 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:09.528+0000 7f22f8381200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:47:09.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:09 smithi037 ceph-mon[27906]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:09.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:09.663+0000 7f8c0bb12200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-mon[32578]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:09.959 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:10.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-mon[32520]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:10.401 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:10.103+0000 7f8c0bb12200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:47:10.401 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:10.199+0000 7f8c0bb12200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:47:10.901 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:10.410+0000 7f8c0bb12200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:47:11.192 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:10 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:10.843+0000 7f22f8381200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:47:11.192 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:11.019+0000 7f22f8381200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:47:11.193 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:11.195+0000 7f22f8381200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:47:11.270 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:10.942+0000 7f8c0bb12200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:47:11.270 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:11.033+0000 7f8c0bb12200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:47:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 -- 172.21.15.37:0/17829227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8102ab0 msgr2=0x7f2ff8102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 --2- 172.21.15.37:0/17829227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8102ab0 0x7f2ff8102ed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2fec009990 tx=0x7f2fec009ca0 comp rx=0 tx=0).stop 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 -- 172.21.15.37:0/17829227 shutdown_connections 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 --2- 172.21.15.37:0/17829227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff8104650 0x7f2ff810af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 --2- 172.21.15.37:0/17829227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8102ab0 0x7f2ff8102ed0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 --2- 172.21.15.37:0/17829227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 -- 172.21.15.37:0/17829227 >> 172.21.15.37:0/17829227 conn(0x7f2ff80fe010 msgr2=0x7f2ff8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 -- 172.21.15.37:0/17829227 shutdown_connections 2024-04-26T01:47:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.348+0000 7f2fffe00700 1 -- 172.21.15.37:0/17829227 wait complete. 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.349+0000 7f2fffe00700 1 Processor -- start 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.349+0000 7f2fffe00700 1 -- start start 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.349+0000 7f2fffe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff8102ab0 0x7f2ff8076790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.349+0000 7f2fffe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.349+0000 7f2fffe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8104650 0x7f2ff8075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2fffe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff810e570 con 0x7f2ff8102ab0 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2fffe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff810e3f0 con 0x7f2ff8103cb0 2024-04-26T01:47:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2fffe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff810e6f0 con 0x7f2ff8104650 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffde00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8104650 0x7f2ff8075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:36110/0 (socket says 172.21.15.37:36110) 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 -- 172.21.15.37:0/2123762973 learned_addr learned my addr 172.21.15.37:0/2123762973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffd400700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff8102ab0 0x7f2ff8076790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 -- 172.21.15.37:0/2123762973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8104650 msgr2=0x7f2ff8075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8104650 0x7f2ff8075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 -- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff8102ab0 msgr2=0x7f2ff8076790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff8102ab0 0x7f2ff8076790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffca00700 1 -- 172.21.15.37:0/2123762973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8075c40 con 0x7f2ff8103cb0 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.350+0000 7f2ffde00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8104650 0x7f2ff8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:47:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.351+0000 7f2ffca00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8074e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e9a0 tx=0x7f2fe400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.351+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe400cbd0 con 0x7f2ff8103cb0 2024-04-26T01:47:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.351+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fe400cd70 con 0x7f2ff8103cb0 2024-04-26T01:47:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.351+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff81a2db0 con 0x7f2ff8103cb0 2024-04-26T01:47:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.351+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fe4018680 con 0x7f2ff8103cb0 2024-04-26T01:47:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.351+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff81a3200 con 0x7f2ff8103cb0 2024-04-26T01:47:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.353+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f2fe4018820 con 0x7f2ff8103cb0 2024-04-26T01:47:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.353+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff8069730 con 0x7f2ff8103cb0 2024-04-26T01:47:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.353+0000 7f2ff6000700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2fe0039890 0x7f2fe003bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.353+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2fe404cbf0 con 0x7f2ff8103cb0 2024-04-26T01:47:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.353+0000 7f2ffd400700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2fe0039890 0x7f2fe003bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.354+0000 7f2ffd400700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2fe0039890 0x7f2fe003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b2e0 tx=0x7f2fec01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.359+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2fe400fc70 con 0x7f2ff8103cb0 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:47:11.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:11.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:11.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:11.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:11 smithi037 ceph-mon[27906]: Reconfiguring daemon mgr.x on smithi037 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:11.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:47:11.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:11.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:11.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:11.492 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-mon[32520]: Reconfiguring daemon mgr.x on smithi037 2024-04-26T01:47:11.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:11.603+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f2ff8062e80 con 0x7f2fe0039890 2024-04-26T01:47:11.610 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:11.272+0000 7f8c0bb12200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:47:11.610 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:11.403+0000 7f8c0bb12200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:47:11.610 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:11.497+0000 7f8c0bb12200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:47:11.610 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:11.612+0000 7f8c0bb12200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:47:11.610 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:11.611 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-mon[32578]: Reconfiguring daemon mgr.x on smithi037 2024-04-26T01:47:11.753 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:11.493+0000 7f22f8381200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:47:11.753 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:11.588+0000 7f22f8381200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:47:12.253 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:11.797+0000 7f22f8381200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:47:12.253 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:11.995+0000 7f22f8381200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:47:12.401 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:11.913+0000 7f8c0bb12200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:47:12.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:12 smithi037 ceph-mon[27906]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:12.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:12.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:12.589 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:12.720 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:12.509+0000 7f8c0bb12200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:47:12.721 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:47:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:47:12.723+0000 7f8c0bb12200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:47:12.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:12 smithi071 ceph-mon[32578]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:12.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:12.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:12.721 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:12.754 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:12 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:12.619+0000 7f22f8381200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:47:12.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:12 smithi045 ceph-mon[32520]: from='client.24119 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:12.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:12.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:12.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:13.364 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:13.056+0000 7f22f8381200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:47:13.364 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:13.155+0000 7f22f8381200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:47:13.364 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:13.366+0000 7f22f8381200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:47:13.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:13.474+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f2fe40178f0 con 0x7f2ff8103cb0 2024-04-26T01:47:13.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:13.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: Standby manager daemon z started 2024-04-26T01:47:13.613 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:13.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:13 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: Standby manager daemon z started 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: Standby manager daemon z started 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/2851944205' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:13.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:13 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:14.252 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:13.909+0000 7f22f8381200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:47:14.252 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:14.004+0000 7f22f8381200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:47:14.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-mon[32520]: mgrmap e14: x(active, since 91s), standbys: z 2024-04-26T01:47:14.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:47:14.504 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:14.254+0000 7f22f8381200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:47:14.504 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:14.393+0000 7f22f8381200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:47:14.504 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:14.493+0000 7f22f8381200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:47:14.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:14 smithi037 ceph-mon[27906]: mgrmap e14: x(active, since 91s), standbys: z 2024-04-26T01:47:14.677 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:47:14.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:14 smithi071 ceph-mon[32578]: mgrmap e14: x(active, since 91s), standbys: z 2024-04-26T01:47:14.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:14 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:47:14.923 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:14.612+0000 7f22f8381200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:47:14.923 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:14.925+0000 7f22f8381200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:47:15.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:15 smithi045 ceph-mon[32520]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:15.753 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:15.550+0000 7f22f8381200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:47:15.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:15 smithi071 ceph-mon[32578]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:15.955 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:15 smithi037 ceph-mon[27906]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:16.253 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:47:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:47:15.772+0000 7f22f8381200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:47:16.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:16.499+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mon.2 v2:172.21.15.45:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f2fe4018b10 con 0x7f2ff8103cb0 2024-04-26T01:47:16.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:47:16.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: Standby manager daemon y started 2024-04-26T01:47:16.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1786041614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]: dispatch 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]: dispatch 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]': finished 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:47:16.654 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:16 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:16.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:47:16.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: Standby manager daemon y started 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1786041614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]: dispatch 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]: dispatch 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]': finished 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:47:16.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:16 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:16.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:47:16.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: Standby manager daemon y started 2024-04-26T01:47:16.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/2000386591' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1786041614' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]: dispatch 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]: dispatch 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d2501dac-396b-46d1-a9ff-b3b157f46fb4"}]': finished 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T01:47:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:16 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:17 smithi071 ceph-mon[32578]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:17 smithi071 ceph-mon[32578]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-04-26T01:47:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:47:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:17 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1145708412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:17.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:17 smithi037 ceph-mon[27906]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:17.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:17 smithi037 ceph-mon[27906]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-04-26T01:47:17.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:47:17.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:17 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1145708412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:18.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:17 smithi045 ceph-mon[32520]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:18.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:17 smithi045 ceph-mon[32520]: mgrmap e15: x(active, since 94s), standbys: z, y 2024-04-26T01:47:18.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:47:18.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:17 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1145708412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:19.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:19 smithi071 ceph-mon[32578]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:19.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:19 smithi037 ceph-mon[27906]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:20.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:19 smithi045 ceph-mon[32520]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:21.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:21 smithi071 ceph-mon[32578]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:21.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:21 smithi037 ceph-mon[27906]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:22.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:21 smithi045 ceph-mon[32520]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:23.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:23 smithi037 ceph-mon[27906]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:23.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:23 smithi071 ceph-mon[32578]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:24.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:23 smithi045 ceph-mon[32520]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:25.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:25 smithi037 ceph-mon[27906]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:25.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:25 smithi071 ceph-mon[32578]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:26.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:25 smithi045 ceph-mon[32520]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:26.900 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:26 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:47:26.900 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:26 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:26.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:26 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:47:26.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:26 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:27.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:26 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T01:47:27.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:26 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:27.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:27 smithi071 ceph-mon[32578]: Deploying daemon osd.0 on smithi037 2024-04-26T01:47:27.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:27 smithi071 ceph-mon[32578]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:27.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:27 smithi037 ceph-mon[27906]: Deploying daemon osd.0 on smithi037 2024-04-26T01:47:27.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:27 smithi037 ceph-mon[27906]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:28.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:27 smithi045 ceph-mon[32520]: Deploying daemon osd.0 on smithi037 2024-04-26T01:47:28.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:27 smithi045 ceph-mon[32520]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:29.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:29 smithi071 ceph-mon[32578]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:29.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:29 smithi037 ceph-mon[27906]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:30.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:29 smithi045 ceph-mon[32520]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:31.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:31 smithi037 ceph-mon[27906]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:31.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:31 smithi071 ceph-mon[32578]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:32.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:31 smithi045 ceph-mon[32520]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:33.832 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:33 smithi037 ceph-mon[27906]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:33.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:33 smithi071 ceph-mon[32578]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:34.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:33 smithi045 ceph-mon[32520]: pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:35.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:35 smithi037 ceph-mon[27906]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:35.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:35.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:35.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:36.078 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:35 smithi045 ceph-mon[32520]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:36.078 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:36.078 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:36.078 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:36.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:35 smithi071 ceph-mon[32578]: pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:36.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:36.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:36.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:38.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:37 smithi071 ceph-mon[32578]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:38.155 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:37 smithi037 ceph-mon[27906]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:38.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:37 smithi045 ceph-mon[32520]: pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:39.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.564+0000 7f2ff6000700 1 -- 172.21.15.37:0/2123762973 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2ff8062e80 con 0x7f2fe0039890 2024-04-26T01:47:39.563 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-04-26T01:47:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.567+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2fe0039890 msgr2=0x7f2fe003bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.567+0000 7f2fffe00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2fe0039890 0x7f2fe003bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00b2e0 tx=0x7f2fec01a040 comp rx=0 tx=0).stop 2024-04-26T01:47:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.567+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 msgr2=0x7f2ff8074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.567+0000 7f2fffe00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8074e30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2fe400e9a0 tx=0x7f2fe400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:47:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.567+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 shutdown_connections 2024-04-26T01:47:39.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.567+0000 7f2fffe00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff8102ab0 0x7f2ff8076790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.568+0000 7f2fffe00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2fe0039890 0x7f2fe003bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.568+0000 7f2fffe00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2ff8104650 0x7f2ff8075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.568+0000 7f2fffe00700 1 --2- 172.21.15.37:0/2123762973 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2ff8103cb0 0x7f2ff8074e30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.568+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 >> 172.21.15.37:0/2123762973 conn(0x7f2ff80fe010 msgr2=0x7f2ff81070d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.568+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 shutdown_connections 2024-04-26T01:47:39.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:47:39.568+0000 7f2fffe00700 1 -- 172.21.15.37:0/2123762973 wait complete. 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:39.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:39 smithi037 ceph-mon[27906]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:47:39.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:39 smithi071 ceph-mon[32578]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:47:39.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:39.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:39.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:39.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:39.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:39 smithi045 ceph-mon[32520]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T01:47:40.291 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T01:47:40.297 INFO:tasks.cephadm:Deploying osd.1 on smithi045 with /dev/vg_nvme/lv_4... 2024-04-26T01:47:40.297 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:47:40.535 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:47:40.535 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:47:40.535 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:47:40.535 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:40.536 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:40.536 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:40.536 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:40.536 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0[42574]: 2024-04-26T01:47:40.384+0000 7f9bee200700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T01:47:40.536 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:47:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0[42574]: 2024-04-26T01:47:40.388+0000 7f9be9000700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:47:40.550 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:47:40.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:47:40.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:47:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:47:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:40 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:40.672 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T01:47:40.672 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T01:47:40.672 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:47:40.673 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:40.673 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:40.673 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:47:40.673 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:40 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:41.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:41 smithi071 ceph-mon[32578]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:47:41.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:41 smithi071 ceph-mon[32578]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:47:41.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:41.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:41.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:41 smithi071 ceph-mon[32578]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:41.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:41 smithi037 ceph-mon[27906]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:47:41.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:41 smithi037 ceph-mon[27906]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:47:41.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:41.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:41.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:41 smithi037 ceph-mon[27906]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:41.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:41 smithi045 ceph-mon[32520]: from='osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:47:41.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:41 smithi045 ceph-mon[32520]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T01:47:41.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:41.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:41.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:41 smithi045 ceph-mon[32520]: pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T01:47:42.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:42 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:47:42.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:42 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:47:42.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:42.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:42 smithi071 ceph-mon[32578]: osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] boot 2024-04-26T01:47:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:42 smithi071 ceph-mon[32578]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:47:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:42.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:42 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:47:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:42 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:47:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:42 smithi037 ceph-mon[27906]: osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] boot 2024-04-26T01:47:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:42 smithi037 ceph-mon[27906]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:47:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:47:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:47:42.720 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-26T01:47:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-26T01:47:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0133355 s, 786 MB/s 2024-04-26T01:47:42.720 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-26T01:47:42.721 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:42 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:47:42.721 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:42 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:47:42.721 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:42.721 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:42 smithi045 ceph-mon[32520]: osd.0 [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] boot 2024-04-26T01:47:42.721 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:42 smithi045 ceph-mon[32520]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T01:47:42.721 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:47:43.352 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_4 2024-04-26T01:47:43.593 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:47:43.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:43 smithi071 ceph-mon[32578]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:47:43.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:43 smithi071 ceph-mon[32578]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:43.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:43 smithi037 ceph-mon[27906]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:47:43.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:43 smithi037 ceph-mon[27906]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:43.708 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:43 smithi045 ceph-mon[32520]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T01:47:43.708 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:43 smithi045 ceph-mon[32520]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 -- 172.21.15.45:0/427410098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0103540 msgr2=0x7f41e0109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 --2- 172.21.15.45:0/427410098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0103540 0x7f41e0109df0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f41d800b0d0 tx=0x7f41d800b3e0 comp rx=0 tx=0).stop 2024-04-26T01:47:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 -- 172.21.15.45:0/427410098 shutdown_connections 2024-04-26T01:47:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 --2- 172.21.15.45:0/427410098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e01019a0 0x7f41e0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 --2- 172.21.15.45:0/427410098 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0103540 0x7f41e0109df0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:44.933 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 --2- 172.21.15.45:0/427410098 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e0102ba0 0x7f41e0103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 -- 172.21.15.45:0/427410098 >> 172.21.15.45:0/427410098 conn(0x7f41e00fcf40 msgr2=0x7f41e00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 -- 172.21.15.45:0/427410098 shutdown_connections 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.932+0000 7f41e6173700 1 -- 172.21.15.45:0/427410098 wait complete. 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 Processor -- start 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 -- start start 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e01019a0 0x7f41e01058e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 0x7f41e0103f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e0103540 0x7f41e01044c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010d520 con 0x7f41e0103540 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010d3a0 con 0x7f41e0102ba0 2024-04-26T01:47:44.934 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41e6173700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f41e010d6a0 con 0x7f41e01019a0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 0x7f41e0103f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41deffd700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 0x7f41e0103f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:36778/0 (socket says 172.21.15.45:36778) 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41deffd700 1 -- 172.21.15.45:0/4150047957 learned_addr learned my addr 172.21.15.45:0/4150047957 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.933+0000 7f41df7fe700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e01019a0 0x7f41e01058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41deffd700 1 -- 172.21.15.45:0/4150047957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e01019a0 msgr2=0x7f41e01058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41deffd700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e01019a0 0x7f41e01058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41deffd700 1 -- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e0103540 msgr2=0x7f41e01044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41deffd700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e0103540 0x7f41e01044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41deffd700 1 -- 172.21.15.45:0/4150047957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e0104d90 con 0x7f41e0102ba0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41deffd700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 0x7f41e0103f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c970 tx=0x7f41d400cd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4007aa0 con 0x7f41e0102ba0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e01a2e20 con 0x7f41e0102ba0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41d4004ce0 con 0x7f41e0102ba0 2024-04-26T01:47:44.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d40055c0 con 0x7f41e0102ba0 2024-04-26T01:47:44.936 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.934+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e01a32a0 con 0x7f41e0102ba0 2024-04-26T01:47:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.935+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e0069730 con 0x7f41e0102ba0 2024-04-26T01:47:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.936+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f41d4007c40 con 0x7f41e0102ba0 2024-04-26T01:47:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.937+0000 7f41dcff9700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f41c80a3bb0 0x7f41c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:47:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.937+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f41d40ce7e0 con 0x7f41e0102ba0 2024-04-26T01:47:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.937+0000 7f41df7fe700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f41c80a3bb0 0x7f41c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:47:44.937 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.938+0000 7f41df7fe700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f41c80a3bb0 0x7f41c80a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006fd0 tx=0x7f41d0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:47:44.940 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:44.942+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41d409ccc0 con 0x7f41e0102ba0 2024-04-26T01:47:45.159 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:47:45.160+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f41e00ff130 con 0x7f41c80a3bb0 2024-04-26T01:47:46.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi037 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi037 to 17094M 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi037 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi037 to 17094M 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:46.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:46.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:46.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi037 2024-04-26T01:47:46.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi037 to 17094M 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:47:46.465 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:48.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:48 smithi037 ceph-mon[27906]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:48.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:48 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/1254527039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]: dispatch 2024-04-26T01:47:48.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:48 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]: dispatch 2024-04-26T01:47:48.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:48 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]': finished 2024-04-26T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:48 smithi037 ceph-mon[27906]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:47:48.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:47:48.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:48 smithi071 ceph-mon[32578]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:48.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:48 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/1254527039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]: dispatch 2024-04-26T01:47:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:48 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]: dispatch 2024-04-26T01:47:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:48 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]': finished 2024-04-26T01:47:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:48 smithi071 ceph-mon[32578]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:47:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:47:48.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:48 smithi045 ceph-mon[32520]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:48 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/1254527039' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]: dispatch 2024-04-26T01:47:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:48 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]: dispatch 2024-04-26T01:47:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:48 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f91f576c-313e-4a6a-9f40-715003e44614"}]': finished 2024-04-26T01:47:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:48 smithi045 ceph-mon[32520]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T01:47:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:47:49.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:49 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/2436724333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:49.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:49 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/2436724333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:49.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:49 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/2436724333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:47:50.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:50 smithi071 ceph-mon[32578]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:50.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:50 smithi037 ceph-mon[27906]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:50.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:50 smithi045 ceph-mon[32520]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:52.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:52 smithi071 ceph-mon[32578]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:52 smithi037 ceph-mon[27906]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:52.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:52 smithi045 ceph-mon[32520]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:54.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:54 smithi045 ceph-mon[32520]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:54.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:54 smithi071 ceph-mon[32578]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:54.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:54 smithi037 ceph-mon[27906]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:55.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:47:55.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:55.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:47:55.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:55.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T01:47:55.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:47:56.351 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:56 smithi045 ceph-mon[32520]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:56.352 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:56 smithi045 ceph-mon[32520]: Deploying daemon osd.1 on smithi045 2024-04-26T01:47:56.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:56 smithi037 ceph-mon[27906]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:56.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:56 smithi037 ceph-mon[27906]: Deploying daemon osd.1 on smithi045 2024-04-26T01:47:56.489 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:56 smithi071 ceph-mon[32578]: pgmap v66: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:56.489 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:56 smithi071 ceph-mon[32578]: Deploying daemon osd.1 on smithi045 2024-04-26T01:47:58.376 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:47:58 smithi045 ceph-mon[32520]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:58.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:47:58 smithi037 ceph-mon[27906]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:47:58.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:47:58 smithi071 ceph-mon[32578]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:00.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:00 smithi037 ceph-mon[27906]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:00.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:00 smithi045 ceph-mon[32520]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:00.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:00 smithi071 ceph-mon[32578]: pgmap v68: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:01.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:01.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:01.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:01.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:01.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:01.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:01.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:01.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:01.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:02.428 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:02 smithi045 ceph-mon[32520]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:02.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:02 smithi037 ceph-mon[27906]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:02.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:02 smithi071 ceph-mon[32578]: pgmap v69: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:03.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:03 smithi071 ceph-mon[32578]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:03.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:03.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:03 smithi045 ceph-mon[32520]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:03.704 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:03 smithi037 ceph-mon[27906]: pgmap v70: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:03.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.804+0000 7f41dcff9700 1 -- 172.21.15.45:0/4150047957 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f41e00ff130 con 0x7f41c80a3bb0 2024-04-26T01:48:03.802 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 1 on host 'smithi045' 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f41c80a3bb0 msgr2=0x7f41c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f41c80a3bb0 0x7f41c80a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006fd0 tx=0x7f41d0008040 comp rx=0 tx=0).stop 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 msgr2=0x7f41e0103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 0x7f41e0103f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f41d400c970 tx=0x7f41d400cd30 comp rx=0 tx=0).stop 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 shutdown_connections 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41e0103540 0x7f41e01044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:03.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f41c80a3bb0 0x7f41c80a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f41e01019a0 0x7f41e01058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 --2- 172.21.15.45:0/4150047957 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f41e0102ba0 0x7f41e0103f80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 >> 172.21.15.45:0/4150047957 conn(0x7f41e00fcf40 msgr2=0x7f41e00fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 shutdown_connections 2024-04-26T01:48:03.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:48:03.806+0000 7f41e6173700 1 -- 172.21.15.45:0/4150047957 wait complete. 2024-04-26T01:48:04.443 DEBUG:teuthology.orchestra.run.smithi045:osd.1> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T01:48:04.445 INFO:tasks.cephadm:Deploying osd.2 on smithi071 with /dev/vg_nvme/lv_4... 2024-04-26T01:48:04.446 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T01:48:04.686 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:48:05.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:05.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:04 smithi045 ceph-mon[32520]: from='osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:48:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:04 smithi045 ceph-mon[32520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:48:05.009 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:05.009 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:05.009 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:05.009 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:04 smithi071 ceph-mon[32578]: from='osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:48:05.009 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:04 smithi071 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:48:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:04 smithi037 ceph-mon[27906]: from='osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:48:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:04 smithi037 ceph-mon[27906]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T01:48:06.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:05 smithi071 ceph-mon[32578]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:06.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:05 smithi071 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:48:06.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:05 smithi071 ceph-mon[32578]: from='osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:48:06.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:05 smithi071 ceph-mon[32578]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:48:06.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:06.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:05 smithi071 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:05 smithi037 ceph-mon[27906]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:05 smithi037 ceph-mon[27906]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:05 smithi037 ceph-mon[27906]: from='osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:05 smithi037 ceph-mon[27906]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:06.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:05 smithi037 ceph-mon[27906]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:48:06.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-mon[32520]: pgmap v71: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-mon[32520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T01:48:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-mon[32520]: from='osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:48:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-mon[32520]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T01:48:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-mon[32520]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:48:06.254 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1[37139]: 2024-04-26T01:48:05.771+0000 7fb33da3e700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T01:48:06.254 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:05 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1[37139]: 2024-04-26T01:48:05.775+0000 7fb339831700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:48:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T01:48:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T01:48:06.644 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-26T01:48:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-26T01:48:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011541 s, 909 MB/s 2024-04-26T01:48:06.644 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-26T01:48:06.912 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:06 smithi045 ceph-mon[32520]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:48:06.912 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:06 smithi045 ceph-mon[32520]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:48:06.912 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:06.912 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:06.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:06 smithi071 ceph-mon[32578]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:48:06.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:06 smithi071 ceph-mon[32578]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:48:06.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:06.952 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:06 smithi037 ceph-mon[27906]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:48:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:06 smithi037 ceph-mon[27906]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T01:48:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:07.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:07.221 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_4 2024-04-26T01:48:07.458 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:48:08.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:48:08.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] boot 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:48:08.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:48:08.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:08.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] boot 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:48:08.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: osd.1 [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] boot 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 -- 172.21.15.71:0/3282975649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 msgr2=0x7f6150101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 --2- 172.21.15.71:0/3282975649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150101dc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6140009990 tx=0x7f6140009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 -- 172.21.15.71:0/3282975649 shutdown_connections 2024-04-26T01:48:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 --2- 172.21.15.71:0/3282975649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150103540 0x7f6150109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 --2- 172.21.15.71:0/3282975649 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150101dc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.866 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 --2- 172.21.15.71:0/3282975649 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 0x7f6150103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 -- 172.21.15.71:0/3282975649 >> 172.21.15.71:0/3282975649 conn(0x7f61500fcf40 msgr2=0x7f61500ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 -- 172.21.15.71:0/3282975649 shutdown_connections 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 -- 172.21.15.71:0/3282975649 wait complete. 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 Processor -- start 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.866+0000 7f6156714700 1 -- start start 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f6156714700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f6156714700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 0x7f61501991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f6156714700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150103540 0x7f61501a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f6156714700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f615010d510 con 0x7f6150103540 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f6156714700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f615010d390 con 0x7f6150102ba0 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f6156714700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f615010d690 con 0x7f61501019a0 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.867 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150198cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:33584/0 (socket says 172.21.15.71:33584) 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 -- 172.21.15.71:0/1475350225 learned_addr learned my addr 172.21.15.71:0/1475350225 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 -- 172.21.15.71:0/1475350225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 msgr2=0x7f61501991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614f7fe700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 0x7f61501991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 0x7f61501991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 -- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150103540 msgr2=0x7f61501a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150103540 0x7f61501a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 -- 172.21.15.71:0/1475350225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61501a0930 con 0x7f61501019a0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614f7fe700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 0x7f61501991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.867+0000 7f614ffff700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150198cb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6140005f50 tx=0x7f61400047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.868+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140027070 con 0x7f61501019a0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.868+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6140022470 con 0x7f61501019a0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.868+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61501a0bc0 con 0x7f61501019a0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.868+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614000f6c0 con 0x7f61501019a0 2024-04-26T01:48:08.868 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.868+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61501a1070 con 0x7f61501019a0 2024-04-26T01:48:08.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.869+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6150069730 con 0x7f61501019a0 2024-04-26T01:48:08.869 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.870+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f614002a070 con 0x7f61501019a0 2024-04-26T01:48:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.870+0000 7f614d7fa700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61380a3c30 0x7f61380a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.871+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f61400cfee0 con 0x7f61501019a0 2024-04-26T01:48:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.871+0000 7f614f7fe700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61380a3c30 0x7f61380a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:08.870 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.872+0000 7f614f7fe700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61380a3c30 0x7f61380a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f615019a1c0 tx=0x7f6144008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:08.872 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:08.874+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6140013070 con 0x7f61501019a0 2024-04-26T01:48:09.076 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:09.078+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f61500ff170 con 0x7f61380a3c30 2024-04-26T01:48:09.077 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:08 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi045 2024-04-26T01:48:09.077 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:08 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi045 to 17093M 2024-04-26T01:48:09.077 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:08 smithi071 ceph-mon[32578]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:48:09.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:08 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi045 2024-04-26T01:48:09.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:08 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi045 to 17093M 2024-04-26T01:48:09.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:08 smithi037 ceph-mon[27906]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:48:09.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:08 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi045 2024-04-26T01:48:09.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:08 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi045 to 17093M 2024-04-26T01:48:09.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:08 smithi045 ceph-mon[32520]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T01:48:09.972 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:09 smithi071 ceph-mon[32578]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:09.972 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:09 smithi071 ceph-mon[32578]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:09.972 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:48:09.972 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:48:09.972 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:10.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:09 smithi037 ceph-mon[27906]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:10.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:09 smithi037 ceph-mon[27906]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:10.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:48:10.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:48:10.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:10.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:09 smithi045 ceph-mon[32520]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:10.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:09 smithi045 ceph-mon[32520]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:48:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:48:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:12.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:11 smithi071 ceph-mon[32578]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:11 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/1127658637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]: dispatch 2024-04-26T01:48:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:11 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]: dispatch 2024-04-26T01:48:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:11 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]': finished 2024-04-26T01:48:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:11 smithi071 ceph-mon[32578]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:48:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:11 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:11 smithi037 ceph-mon[27906]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:11 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/1127658637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]: dispatch 2024-04-26T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:11 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]: dispatch 2024-04-26T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:11 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]': finished 2024-04-26T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:11 smithi037 ceph-mon[27906]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:48:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:11 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:11 smithi045 ceph-mon[32520]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:11 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/1127658637' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]: dispatch 2024-04-26T01:48:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:11 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]: dispatch 2024-04-26T01:48:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:11 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a90c81da-6303-4de4-8556-bd4ba8819380"}]': finished 2024-04-26T01:48:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:11 smithi045 ceph-mon[32520]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T01:48:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:11 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:13.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:12 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/4081616977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:13.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:12 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/4081616977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:13.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:12 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/4081616977' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:14.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:13 smithi037 ceph-mon[27906]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:14.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:13 smithi071 ceph-mon[32578]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:14.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:13 smithi045 ceph-mon[32520]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:16.074 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:15 smithi071 ceph-mon[32578]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:16.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:15 smithi037 ceph-mon[27906]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:16.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:15 smithi045 ceph-mon[32520]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:17.885 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:17 smithi071 ceph-mon[32578]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:18.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:17 smithi037 ceph-mon[27906]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:18.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:17 smithi045 ceph-mon[32520]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:20.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:19 smithi071 ceph-mon[32578]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:20.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:19 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:48:20.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:19 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:20.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:19 smithi071 ceph-mon[32578]: Deploying daemon osd.2 on smithi071 2024-04-26T01:48:20.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:19 smithi037 ceph-mon[27906]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:20.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:48:20.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:20.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:19 smithi037 ceph-mon[27906]: Deploying daemon osd.2 on smithi071 2024-04-26T01:48:20.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:19 smithi045 ceph-mon[32520]: pgmap v83: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:19 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T01:48:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:19 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:19 smithi045 ceph-mon[32520]: Deploying daemon osd.2 on smithi071 2024-04-26T01:48:21.986 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:21 smithi071 ceph-mon[32578]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:22.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:21 smithi037 ceph-mon[27906]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:22.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:21 smithi045 ceph-mon[32520]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:23.972 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:23 smithi071 ceph-mon[32578]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:24.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:23 smithi037 ceph-mon[27906]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:24.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:23 smithi045 ceph-mon[32520]: pgmap v85: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:25.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:25.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:25.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:25.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:25 smithi037 ceph-mon[27906]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:25.487 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:25.488 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:25.488 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:25.488 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:25 smithi071 ceph-mon[32578]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:25 smithi045 ceph-mon[32520]: pgmap v86: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:27.412 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.413+0000 7f614d7fa700 1 -- 172.21.15.71:0/1475350225 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f61500ff170 con 0x7f61380a3c30 2024-04-26T01:48:27.412 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 2 on host 'smithi071' 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61380a3c30 msgr2=0x7f61380a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61380a3c30 0x7f61380a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f615019a1c0 tx=0x7f6144008040 comp rx=0 tx=0).stop 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 msgr2=0x7f6150198cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150198cb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6140005f50 tx=0x7f61400047b0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 shutdown_connections 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6150103540 0x7f61501a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61380a3c30 0x7f61380a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61501019a0 0x7f6150198cb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.414 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 --2- 172.21.15.71:0/1475350225 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6150102ba0 0x7f61501991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 >> 172.21.15.71:0/1475350225 conn(0x7f61500fcf40 msgr2=0x7f61500fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 shutdown_connections 2024-04-26T01:48:27.415 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:48:27.416+0000 7f6156714700 1 -- 172.21.15.71:0/1475350225 wait complete. 2024-04-26T01:48:28.011 DEBUG:teuthology.orchestra.run.smithi071:osd.2> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T01:48:28.013 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-04-26T01:48:28.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:48:28.316 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2[37694]: 2024-04-26T01:48:28.098+0000 7f40526897c0 -1 osd.2 0 log_to_monitors true 2024-04-26T01:48:28.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:28.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:28.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:28.317 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.317 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.317 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.317 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:28.474 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: pgmap v87: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:29.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:29 smithi037 ceph-mon[27906]: from='osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:48:29.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:29 smithi037 ceph-mon[27906]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:48:29.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:29 smithi071 ceph-mon[32578]: from='osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:48:29.265 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:29 smithi071 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:48:29.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:29 smithi045 ceph-mon[32520]: from='osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:48:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:29 smithi045 ceph-mon[32520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T01:48:30.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:30 smithi037 ceph-mon[27906]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:30.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:30 smithi037 ceph-mon[27906]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:48:30.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:30 smithi037 ceph-mon[27906]: from='osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:48:30.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:30 smithi037 ceph-mon[27906]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:48:30.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:30.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:30 smithi037 ceph-mon[27906]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:48:30.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:30 smithi071 ceph-mon[32578]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:30.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:30 smithi071 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:48:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:30 smithi071 ceph-mon[32578]: from='osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:48:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:30 smithi071 ceph-mon[32578]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:48:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:30 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:30 smithi071 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:48:30.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:30 smithi045 ceph-mon[32520]: pgmap v88: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:30 smithi045 ceph-mon[32520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T01:48:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:30 smithi045 ceph-mon[32520]: from='osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:48:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:30 smithi045 ceph-mon[32520]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T01:48:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:30 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:30 smithi045 ceph-mon[32520]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:48:31.305 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:48:31.305 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:48:31.305 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='osd.2 ' entity='osd.2' 2024-04-26T01:48:31.306 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:48:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:48:31.307 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T01:48:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T01:48:31.307 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0371875 s, 282 MB/s 2024-04-26T01:48:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T01:48:31.401 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2[37694]: 2024-04-26T01:48:31.002+0000 7f4047cee700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T01:48:31.402 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2[37694]: 2024-04-26T01:48:31.227+0000 7f403fad9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='osd.2 ' entity='osd.2' 2024-04-26T01:48:31.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:31.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='osd.2 ' entity='osd.2' 2024-04-26T01:48:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:32.111 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-04-26T01:48:32.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:32 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:48:32.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:32 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:48:32.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:32 smithi037 ceph-mon[27906]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:32.193 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:32 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi071 2024-04-26T01:48:32.193 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:32 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-26T01:48:32.193 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:32 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:32.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:32 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:48:32.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:32 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:48:32.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:32 smithi071 ceph-mon[32578]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:32.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:32 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi071 2024-04-26T01:48:32.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:32 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-26T01:48:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:32 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:32.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:32 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:48:32.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:32 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:48:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:32 smithi045 ceph-mon[32520]: pgmap v91: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:32 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi071 2024-04-26T01:48:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:32 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi071 to 17088M 2024-04-26T01:48:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:32 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:32.570 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:48:33.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:33 smithi071 ceph-mon[32578]: osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] boot 2024-04-26T01:48:33.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:33 smithi071 ceph-mon[32578]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:48:33.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:33 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:33.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:33 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:48:33.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:33 smithi037 ceph-mon[27906]: osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] boot 2024-04-26T01:48:33.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:33 smithi037 ceph-mon[27906]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:48:33.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:33 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:33.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:33 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:48:33.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:33 smithi045 ceph-mon[32520]: osd.2 [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] boot 2024-04-26T01:48:33.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:33 smithi045 ceph-mon[32520]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T01:48:33.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:33 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:48:33.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:33 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:48:34.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 ceph-mon[27906]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:34.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:48:34.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 ceph-mon[27906]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:48:34.185 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:48:34.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 ceph-mon[32578]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:34.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:48:34.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 ceph-mon[32578]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:48:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:48:34.415 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 ceph-mon[32520]: pgmap v93: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T01:48:34.415 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T01:48:34.415 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 ceph-mon[32520]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T01:48:34.415 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47087]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47087]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47087]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47090]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47090]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47090]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.445 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47090]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47100]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47100]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47100]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47100]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47103]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47103]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47103]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:34 smithi037 sudo[47103]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41270]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41270]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41270]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41270]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41273]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41273]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41273]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.714 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41273]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 -- 172.21.15.37:0/279578447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 msgr2=0x7efccc104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:34.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 --2- 172.21.15.37:0/279578447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc104140 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcb8009990 tx=0x7efcb8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 -- 172.21.15.37:0/279578447 shutdown_connections 2024-04-26T01:48:34.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 --2- 172.21.15.37:0/279578447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efccc104680 0x7efccc10af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 --2- 172.21.15.37:0/279578447 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efccc102ae0 0x7efccc102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 --2- 172.21.15.37:0/279578447 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc104140 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.725+0000 7efcd4000700 1 -- 172.21.15.37:0/279578447 >> 172.21.15.37:0/279578447 conn(0x7efccc0fe080 msgr2=0x7efccc1004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:48:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.726+0000 7efcd4000700 1 -- 172.21.15.37:0/279578447 shutdown_connections 2024-04-26T01:48:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.726+0000 7efcd4000700 1 -- 172.21.15.37:0/279578447 wait complete. 2024-04-26T01:48:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.727+0000 7efcd4000700 1 Processor -- start 2024-04-26T01:48:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.727+0000 7efcd4000700 1 -- start start 2024-04-26T01:48:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd4000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efccc102ae0 0x7efccc19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd4000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc19b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd4000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efccc104680 0x7efccc1a2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd4000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efccc10e550 con 0x7efccc102ae0 2024-04-26T01:48:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd4000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efccc10e3d0 con 0x7efccc104680 2024-04-26T01:48:34.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd4000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efccc10e6d0 con 0x7efccc103ce0 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd1600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efccc102ae0 0x7efccc19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd0c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc19b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd0c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc19b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38120/0 (socket says 172.21.15.37:38120) 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd0c00700 1 -- 172.21.15.37:0/3798075449 learned_addr learned my addr 172.21.15.37:0/3798075449 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.728+0000 7efcd2000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efccc104680 0x7efccc1a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd0c00700 1 -- 172.21.15.37:0/3798075449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efccc104680 msgr2=0x7efccc1a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd0c00700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efccc104680 0x7efccc1a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd0c00700 1 -- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efccc102ae0 msgr2=0x7efccc19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd0c00700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efccc102ae0 0x7efccc19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd0c00700 1 -- 172.21.15.37:0/3798075449 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efccc1a2a10 con 0x7efccc103ce0 2024-04-26T01:48:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd0c00700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc19b340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efcb8000c00 tx=0x7efcb800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8026070 con 0x7efccc103ce0 2024-04-26T01:48:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcb800be90 con 0x7efccc103ce0 2024-04-26T01:48:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcb8017650 con 0x7efccc103ce0 2024-04-26T01:48:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.729+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efccc1a2ca0 con 0x7efccc103ce0 2024-04-26T01:48:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.730+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efccc1a3150 con 0x7efccc103ce0 2024-04-26T01:48:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.731+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efccc069730 con 0x7efccc103ce0 2024-04-26T01:48:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.733+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7efcb8029070 con 0x7efccc103ce0 2024-04-26T01:48:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.734+0000 7efcca000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efcb40a3c30 0x7efcb40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:48:34.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.734+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3237+0+0 (secure 0 0 0) 0x7efcb80d00e0 con 0x7efccc103ce0 2024-04-26T01:48:34.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.734+0000 7efcd1600700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efcb40a3c30 0x7efcb40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:48:34.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.735+0000 7efcd1600700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efcb40a3c30 0x7efcb40a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efccc103b40 tx=0x7efcc000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:48:34.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.736+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efcb8030080 con 0x7efccc103ce0 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41468]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41468]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41468]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41468]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41471]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41471]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41471]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41471]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.902 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41462]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41462]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41462]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41462]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41465]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41465]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41465]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:34.903 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 01:48:34 smithi071 sudo[41465]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:34.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:34.981+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7efccc076390 con 0x7efcb40a3c30 2024-04-26T01:48:35.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41276]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41276]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41276]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41276]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41279]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41279]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41279]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T01:48:35.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:34 smithi045 sudo[41279]: pam_unix(sudo:session): session closed for user root 2024-04-26T01:48:35.281 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.282 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:48:35.283 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:48:35.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:35.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:48:35.423+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7efcb8009d10 con 0x7efccc103ce0 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.424 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:48:35.425 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:36.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:36 smithi037 ceph-mon[27906]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:36.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:36 smithi037 ceph-mon[27906]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:36.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:36 smithi037 ceph-mon[27906]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:48:36.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:36 smithi037 ceph-mon[27906]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T01:48:36.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:36 smithi071 ceph-mon[32578]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:36.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:36 smithi071 ceph-mon[32578]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:36.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:36 smithi071 ceph-mon[32578]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:48:36.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:36 smithi071 ceph-mon[32578]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T01:48:36.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:36 smithi045 ceph-mon[32520]: pgmap v96: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:36.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:36 smithi045 ceph-mon[32520]: from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:48:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:36 smithi045 ceph-mon[32520]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T01:48:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:36 smithi045 ceph-mon[32520]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T01:48:38.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:38 smithi071 ceph-mon[32578]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:38.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:38 smithi037 ceph-mon[27906]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:38.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:38 smithi045 ceph-mon[32520]: pgmap v98: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:39.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:39 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1901796462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "950c30ee-de7f-461e-89b8-5218e34caf03"}]: dispatch 2024-04-26T01:48:39.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:39 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1901796462' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950c30ee-de7f-461e-89b8-5218e34caf03"}]': finished 2024-04-26T01:48:39.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:39 smithi037 ceph-mon[27906]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T01:48:39.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:39 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:48:39.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:39 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1901796462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "950c30ee-de7f-461e-89b8-5218e34caf03"}]: dispatch 2024-04-26T01:48:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:39 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1901796462' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950c30ee-de7f-461e-89b8-5218e34caf03"}]': finished 2024-04-26T01:48:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:39 smithi045 ceph-mon[32520]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T01:48:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:39 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:48:39.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:39 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1901796462' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "950c30ee-de7f-461e-89b8-5218e34caf03"}]: dispatch 2024-04-26T01:48:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:39 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1901796462' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "950c30ee-de7f-461e-89b8-5218e34caf03"}]': finished 2024-04-26T01:48:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:39 smithi071 ceph-mon[32578]: osdmap e22: 4 total, 3 up, 4 in 2024-04-26T01:48:39.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:39 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:48:40.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:40 smithi037 ceph-mon[27906]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:40.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:40 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2716487538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:40.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:40 smithi045 ceph-mon[32520]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:40.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:40 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2716487538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:40.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:40 smithi071 ceph-mon[32578]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:40.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:40 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2716487538' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:48:42.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:42 smithi037 ceph-mon[27906]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:42 smithi045 ceph-mon[32520]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:42.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:42 smithi071 ceph-mon[32578]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:44.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:44 smithi037 ceph-mon[27906]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:44 smithi045 ceph-mon[32520]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:44.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:44 smithi071 ceph-mon[32578]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:46.441 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:46 smithi037 ceph-mon[27906]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:46.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:46 smithi045 ceph-mon[32520]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:46.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:46 smithi071 ceph-mon[32578]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:48.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:48 smithi037 ceph-mon[27906]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:48.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:48 smithi045 ceph-mon[32520]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:48.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:48 smithi071 ceph-mon[32578]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:50 smithi037 ceph-mon[27906]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:48:50.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:50.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:50 smithi045 ceph-mon[32520]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:50.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:48:50.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:50.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:50 smithi071 ceph-mon[32578]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T01:48:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:48:51.364 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:51 smithi037 ceph-mon[27906]: Deploying daemon osd.3 on smithi037 2024-04-26T01:48:51.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:51 smithi045 ceph-mon[32520]: Deploying daemon osd.3 on smithi037 2024-04-26T01:48:51.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:51 smithi071 ceph-mon[32578]: Deploying daemon osd.3 on smithi037 2024-04-26T01:48:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:52 smithi037 ceph-mon[27906]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:52.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:52 smithi045 ceph-mon[32520]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:52.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:52 smithi071 ceph-mon[32578]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:54.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:54 smithi037 ceph-mon[27906]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:54.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:54 smithi045 ceph-mon[32520]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:54.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:54 smithi071 ceph-mon[32578]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:56.453 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:56 smithi071 ceph-mon[32578]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:56.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:56 smithi037 ceph-mon[27906]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:56.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:56 smithi045 ceph-mon[32520]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:58.452 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:58 smithi037 ceph-mon[27906]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:58.452 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:58.452 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:58.452 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:58.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:58 smithi045 ceph-mon[32520]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:58.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:58.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:58.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:58.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:58 smithi071 ceph-mon[32578]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:58.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:48:58.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:58.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:48:59.420 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:48:59 smithi037 ceph-mon[27906]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:59.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:48:59 smithi071 ceph-mon[32578]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:48:59.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:48:59 smithi045 ceph-mon[32520]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:01.616 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:01 smithi037 ceph-mon[27906]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:01.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:01 smithi045 ceph-mon[32520]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:01.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:01 smithi071 ceph-mon[32578]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:03 smithi037 ceph-mon[27906]: from='osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:49:03.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:03 smithi037 ceph-mon[27906]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:49:03.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.347+0000 7efcca000700 1 -- 172.21.15.37:0/3798075449 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7efccc076390 con 0x7efcb40a3c30 2024-04-26T01:49:03.347 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-04-26T01:49:03.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efcb40a3c30 msgr2=0x7efcb40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efcb40a3c30 0x7efcb40a60f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efccc103b40 tx=0x7efcc000b410 comp rx=0 tx=0).stop 2024-04-26T01:49:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 msgr2=0x7efccc19b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc19b340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7efcb8000c00 tx=0x7efcb800f690 comp rx=0 tx=0).stop 2024-04-26T01:49:03.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 shutdown_connections 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efccc102ae0 0x7efccc19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efcb40a3c30 0x7efcb40a60f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efccc104680 0x7efccc1a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 --2- 172.21.15.37:0/3798075449 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efccc103ce0 0x7efccc19b340 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.351+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 >> 172.21.15.37:0/3798075449 conn(0x7efccc0fe080 msgr2=0x7efccc074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.352+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 shutdown_connections 2024-04-26T01:49:03.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:03.352+0000 7efcd4000700 1 -- 172.21.15.37:0/3798075449 wait complete. 2024-04-26T01:49:03.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:03.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:03.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:03 smithi071 ceph-mon[32578]: from='osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:49:03.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:03 smithi071 ceph-mon[32578]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:49:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:03 smithi045 ceph-mon[32520]: from='osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:49:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:03 smithi045 ceph-mon[32520]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T01:49:04.154 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T01:49:04.159 INFO:tasks.cephadm:Deploying osd.4 on smithi045 with /dev/vg_nvme/lv_3... 2024-04-26T01:49:04.159 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:04.413 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:49:04.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:04.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:49:04.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:49:04.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:04.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:05 smithi045 ceph-mon[32520]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:49:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:05 smithi045 ceph-mon[32520]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:49:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:05 smithi045 ceph-mon[32520]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T01:49:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:05 smithi071 ceph-mon[32578]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:49:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:05 smithi071 ceph-mon[32578]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:49:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:05 smithi071 ceph-mon[32578]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T01:49:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-mon[27906]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:49:05.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-mon[27906]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T01:49:05.407 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-mon[27906]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T01:49:05.408 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:05.408 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3[49397]: 2024-04-26T01:49:05.111+0000 7fa273c00700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T01:49:05.408 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 01:49:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3[49397]: 2024-04-26T01:49:05.148+0000 7fa269000700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:49:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:49:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:49:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:06.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:06.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: from='osd.3 ' entity='osd.3' 2024-04-26T01:49:06.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] boot 2024-04-26T01:49:06.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T01:49:06.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: from='osd.3 ' entity='osd.3' 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] boot 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T01:49:06.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: from='osd.3 ' entity='osd.3' 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: osd.3 [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] boot 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T01:49:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:49:06.458 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:49:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:49:06.459 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-26T01:49:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-26T01:49:06.459 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00883923 s, 1.2 GB/s 2024-04-26T01:49:06.459 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-26T01:49:07.012 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_3 2024-04-26T01:49:07.256 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:49:07.669 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:07 smithi045 ceph-mon[32520]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:07.669 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:07 smithi045 ceph-mon[32520]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:49:07.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:07 smithi037 ceph-mon[27906]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:07.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:07 smithi037 ceph-mon[27906]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:49:07.710 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:07 smithi071 ceph-mon[32578]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:07.710 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:07 smithi071 ceph-mon[32578]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T01:49:08.676 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:08 smithi045 ceph-mon[32520]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:49:08.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:08 smithi037 ceph-mon[27906]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 -- 172.21.15.45:0/3096949072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844108790 msgr2=0x7f184410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 --2- 172.21.15.45:0/3096949072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844108790 0x7f184410ac00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1834009990 tx=0x7f1834009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 -- 172.21.15.45:0/3096949072 shutdown_connections 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 --2- 172.21.15.45:0/3096949072 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844108790 0x7f184410ac00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 --2- 172.21.15.45:0/3096949072 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844103530 0x7f1844105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 --2- 172.21.15.45:0/3096949072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1844105e60 0x7f1844108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 -- 172.21.15.45:0/3096949072 >> 172.21.15.45:0/3096949072 conn(0x7f18440fcf40 msgr2=0x7f18440ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:08.709 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 -- 172.21.15.45:0/3096949072 shutdown_connections 2024-04-26T01:49:08.710 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.711+0000 7f184c38b700 1 -- 172.21.15.45:0/3096949072 wait complete. 2024-04-26T01:49:08.711 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 Processor -- start 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 -- start start 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 0x7f1844198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1844105e60 0x7f1844199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844108790 0x7f18441a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f184410e600 con 0x7f1844105e60 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f184410e480 con 0x7f1844103530 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184c38b700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f184410e780 con 0x7f1844108790 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184a127700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 0x7f1844198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184a127700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 0x7f1844198d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:45230/0 (socket says 172.21.15.45:45230) 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184a127700 1 -- 172.21.15.45:0/125650588 learned_addr learned my addr 172.21.15.45:0/125650588 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.712+0000 7f184a928700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844108790 0x7f18441a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:08.712 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a127700 1 -- 172.21.15.45:0/125650588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844108790 msgr2=0x7f18441a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a127700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844108790 0x7f18441a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a127700 1 -- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1844105e60 msgr2=0x7f1844199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a127700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1844105e60 0x7f1844199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a127700 1 -- 172.21.15.45:0/125650588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18441a0990 con 0x7f1844103530 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a928700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844108790 0x7f18441a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184a127700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 0x7f1844198d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f183c00ba50 tx=0x7f183c00be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c019070 con 0x7f1844103530 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18441a0c80 con 0x7f1844103530 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c014440 con 0x7f1844103530 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.713+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c0075d0 con 0x7f1844103530 2024-04-26T01:49:08.713 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.714+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a1170 con 0x7f1844103530 2024-04-26T01:49:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.715+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1844069730 con 0x7f1844103530 2024-04-26T01:49:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.715+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f183c01c070 con 0x7f1844103530 2024-04-26T01:49:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.716+0000 7f18437fe700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f182c0a3c40 0x7f182c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.716+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f183c0ce880 con 0x7f1844103530 2024-04-26T01:49:08.714 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.717+0000 7f1849926700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f182c0a3c40 0x7f182c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:08.715 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.718+0000 7f1849926700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f182c0a3c40 0x7f182c0a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18380097b0 tx=0x7f1838006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:08.718 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.721+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f183c09c3e0 con 0x7f1844103530 2024-04-26T01:49:08.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:08 smithi071 ceph-mon[32578]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T01:49:08.935 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:08.937+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1844062e80 con 0x7f182c0a3c40 2024-04-26T01:49:09.725 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:49:09.725 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:49:09.725 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:49:09.725 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:49:09.725 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:49:09.725 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:09.726 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:09.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:09.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: pgmap v121: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:09.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:09.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:09.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:10.684 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:10 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi037 2024-04-26T01:49:10.685 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:10 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi037 to 8547M 2024-04-26T01:49:10.685 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:10 smithi045 ceph-mon[32520]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T01:49:10.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:10 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi037 2024-04-26T01:49:10.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:10 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi037 to 8547M 2024-04-26T01:49:10.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:10 smithi071 ceph-mon[32578]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T01:49:10.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:10 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi037 2024-04-26T01:49:10.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:10 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi037 to 8547M 2024-04-26T01:49:10.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:10 smithi037 ceph-mon[27906]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T01:49:11.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:11 smithi045 ceph-mon[32520]: pgmap v124: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:49:11.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:11 smithi071 ceph-mon[32578]: pgmap v124: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:49:11.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:11 smithi037 ceph-mon[27906]: pgmap v124: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T01:49:12.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:12 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/4210781098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]: dispatch 2024-04-26T01:49:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:12 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]: dispatch 2024-04-26T01:49:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:12 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]': finished 2024-04-26T01:49:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:12 smithi037 ceph-mon[27906]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T01:49:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:12 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/4192123452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:49:12.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:12 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/4210781098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]: dispatch 2024-04-26T01:49:12.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:12 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]: dispatch 2024-04-26T01:49:12.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:12 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]': finished 2024-04-26T01:49:12.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:12 smithi045 ceph-mon[32520]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T01:49:12.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:12.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:12 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/4192123452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:49:12.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:12 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/4210781098' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]: dispatch 2024-04-26T01:49:12.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:12 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]: dispatch 2024-04-26T01:49:12.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:12 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b70dc87f-17e7-44e4-b5d1-80abdcc91aed"}]': finished 2024-04-26T01:49:12.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:12 smithi071 ceph-mon[32578]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T01:49:12.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:12.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:12 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/4192123452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:49:13.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:13 smithi071 ceph-mon[32578]: pgmap v126: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T01:49:13.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:13 smithi037 ceph-mon[27906]: pgmap v126: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T01:49:14.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:13 smithi045 ceph-mon[32520]: pgmap v126: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T01:49:15.741 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:15 smithi045 ceph-mon[32520]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:15.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:15 smithi071 ceph-mon[32578]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:15.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:15 smithi037 ceph-mon[27906]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:17.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:17 smithi045 ceph-mon[32520]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:17 smithi071 ceph-mon[32578]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:17.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:17 smithi037 ceph-mon[27906]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:19.627 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:19 smithi045 ceph-mon[32520]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:19.627 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:19 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:49:19.627 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:19 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:19.627 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:19 smithi045 ceph-mon[32520]: Deploying daemon osd.4 on smithi045 2024-04-26T01:49:19.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:19 smithi071 ceph-mon[32578]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:19.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:19 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:49:19.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:19 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:19.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:19 smithi071 ceph-mon[32578]: Deploying daemon osd.4 on smithi045 2024-04-26T01:49:19.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:19 smithi037 ceph-mon[27906]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:19.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T01:49:19.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:19.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:19 smithi037 ceph-mon[27906]: Deploying daemon osd.4 on smithi045 2024-04-26T01:49:21.711 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:21 smithi045 ceph-mon[32520]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:21.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:21 smithi071 ceph-mon[32578]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:21.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:21 smithi037 ceph-mon[27906]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:23.692 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:23 smithi045 ceph-mon[32520]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:23.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:23 smithi071 ceph-mon[32578]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:23.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:23 smithi037 ceph-mon[27906]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:25.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:25.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:25.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:25.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:25.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:25.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:25.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:25.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:26.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:25 smithi037 ceph-mon[27906]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:26.235 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:25 smithi045 ceph-mon[32520]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:26.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:25 smithi071 ceph-mon[32578]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:27.422 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.424+0000 7f18437fe700 1 -- 172.21.15.45:0/125650588 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1844062e80 con 0x7f182c0a3c40 2024-04-26T01:49:27.422 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 4 on host 'smithi045' 2024-04-26T01:49:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f182c0a3c40 msgr2=0x7f182c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f182c0a3c40 0x7f182c0a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f18380097b0 tx=0x7f1838006d20 comp rx=0 tx=0).stop 2024-04-26T01:49:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 msgr2=0x7f1844198d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:27.424 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 0x7f1844198d30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f183c00ba50 tx=0x7f183c00be10 comp rx=0 tx=0).stop 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 shutdown_connections 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f182c0a3c40 0x7f182c0a6100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1844108790 0x7f18441a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1844103530 0x7f1844198d30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 --2- 172.21.15.45:0/125650588 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1844105e60 0x7f1844199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 >> 172.21.15.45:0/125650588 conn(0x7f18440fcf40 msgr2=0x7f1844103fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 shutdown_connections 2024-04-26T01:49:27.425 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:49:27.426+0000 7f184c38b700 1 -- 172.21.15.45:0/125650588 wait complete. 2024-04-26T01:49:28.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: from='osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:49:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:27 smithi045 ceph-mon[32520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:49:28.037 DEBUG:teuthology.orchestra.run.smithi045:osd.4> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T01:49:28.039 INFO:tasks.cephadm:Deploying osd.5 on smithi071 with /dev/vg_nvme/lv_3... 2024-04-26T01:49:28.040 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T01:49:28.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: from='osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:49:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:27 smithi037 ceph-mon[27906]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:49:28.272 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:49:28.393 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:28.393 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.393 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.394 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.394 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:28.394 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: from='osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:49:28.394 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:27 smithi071 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T01:49:29.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-mon[32520]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:29.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-mon[32520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:49:29.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-mon[32520]: from='osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:49:29.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-mon[32520]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:49:29.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:29.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-mon[32520]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:49:29.754 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T01:49:29.724+0000 7f24943d8700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T01:49:29.755 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 01:49:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T01:49:29.732+0000 7f24901cb700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:49:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:29 smithi071 ceph-mon[32578]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:29 smithi071 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:49:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:29 smithi071 ceph-mon[32578]: from='osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:49:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:29 smithi071 ceph-mon[32578]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:49:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:29 smithi071 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:49:30.103 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T01:49:30.103 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T01:49:30.103 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-26T01:49:30.103 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-26T01:49:30.103 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126817 s, 827 MB/s 2024-04-26T01:49:30.103 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-26T01:49:30.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:29 smithi037 ceph-mon[27906]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:30.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:29 smithi037 ceph-mon[27906]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T01:49:30.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:29 smithi037 ceph-mon[27906]: from='osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:49:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:29 smithi037 ceph-mon[27906]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T01:49:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:29 smithi037 ceph-mon[27906]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:49:30.696 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_3 2024-04-26T01:49:30.933 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:49:31.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:30 smithi045 ceph-mon[32520]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:49:31.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:30 smithi045 ceph-mon[32520]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T01:49:31.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:30 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:30 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.055 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:30 smithi071 ceph-mon[32578]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:49:31.055 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:30 smithi071 ceph-mon[32578]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T01:49:31.055 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:30 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.055 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:30 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:30 smithi037 ceph-mon[27906]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:49:31.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:30 smithi037 ceph-mon[27906]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T01:49:31.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] boot 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:31.939 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi045 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi045 to 8546M 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:31.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:49:32.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:49:32.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:32.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] boot 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi045 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi045 to 8546M 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:32.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:32.006 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:32.006 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:49:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:49:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-26T01:49:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: osd.4 [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] boot 2024-04-26T01:49:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T01:49:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi045 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi045 to 8546M 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:32.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:32.396 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.396+0000 7f810d958700 1 -- 172.21.15.71:0/593800341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 msgr2=0x7f8108101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.396+0000 7f810d958700 1 --2- 172.21.15.71:0/593800341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f8108101db0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f80fc009990 tx=0x7f80fc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 -- 172.21.15.71:0/593800341 shutdown_connections 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 --2- 172.21.15.71:0/593800341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8108103530 0x7f8108109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 --2- 172.21.15.71:0/593800341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f8108101db0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 --2- 172.21.15.71:0/593800341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108102b90 0x7f8108102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 -- 172.21.15.71:0/593800341 >> 172.21.15.71:0/593800341 conn(0x7f81080fcf30 msgr2=0x7f81080ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 -- 172.21.15.71:0/593800341 shutdown_connections 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 -- 172.21.15.71:0/593800341 wait complete. 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 Processor -- start 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.397+0000 7f810d958700 1 -- start start 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f810806bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8108102b90 0x7f810819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:32.397 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108103530 0x7f81081a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f810810d500 con 0x7f8108102b90 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f810810d380 con 0x7f8108103530 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f810810d680 con 0x7f8108101990 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f810806bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f810806bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:57448/0 (socket says 172.21.15.71:57448) 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 -- 172.21.15.71:0/960341094 learned_addr learned my addr 172.21.15.71:0/960341094 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f81077fe700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108103530 0x7f81081a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 -- 172.21.15.71:0/960341094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108103530 msgr2=0x7f81081a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108103530 0x7f81081a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 -- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8108102b90 msgr2=0x7f810819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8108102b90 0x7f810819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:32.398 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 -- 172.21.15.71:0/960341094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8108197820 con 0x7f8108101990 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f81077fe700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108103530 0x7f81081a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f8106ffd700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f810806bb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005f50 tx=0x7f80fc0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc027070 con 0x7f8108101990 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8108197ab0 con 0x7f8108101990 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80fc022470 con 0x7f8108101990 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80fc00f6c0 con 0x7f8108101990 2024-04-26T01:49:32.399 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.398+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8108198060 con 0x7f8108101990 2024-04-26T01:49:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.399+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8108069730 con 0x7f8108101990 2024-04-26T01:49:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.400+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f80fc02a070 con 0x7f8108101990 2024-04-26T01:49:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.401+0000 7f810c956700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f80f00a3cc0 0x7f80f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.401+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f80fc0d0750 con 0x7f8108101990 2024-04-26T01:49:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.401+0000 7f81067fc700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f80f00a3cc0 0x7f80f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:32.400 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.402+0000 7f81067fc700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f80f00a3cc0 0x7f80f00a6180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f810806c090 tx=0x7f80f8006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:32.403 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.406+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80fc013070 con 0x7f8108101990 2024-04-26T01:49:32.613 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:32.615+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f81081089c0 con 0x7f80f00a3cc0 2024-04-26T01:49:32.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:32 smithi071 ceph-mon[32578]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T01:49:32.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:32 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:49:32.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:32 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:49:32.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:32 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:33.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:32 smithi037 ceph-mon[27906]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T01:49:33.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:32 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:49:33.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:32 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:49:33.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:32 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:33.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:32 smithi045 ceph-mon[32520]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T01:49:33.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:32 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:49:33.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:32 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:49:33.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:32 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:34.022 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:33 smithi071 ceph-mon[32578]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:34.023 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:33 smithi071 ceph-mon[32578]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:49:34.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:33 smithi037 ceph-mon[27906]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:34.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:33 smithi037 ceph-mon[27906]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:49:34.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:33 smithi045 ceph-mon[32520]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:34.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:33 smithi045 ceph-mon[32520]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:49:36.056 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:35 smithi045 ceph-mon[32520]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:36.057 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:35 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/999835689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]: dispatch 2024-04-26T01:49:36.057 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:35 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]: dispatch 2024-04-26T01:49:36.057 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:35 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]': finished 2024-04-26T01:49:36.057 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:35 smithi045 ceph-mon[32520]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T01:49:36.057 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:35 smithi071 ceph-mon[32578]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:35 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/999835689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]: dispatch 2024-04-26T01:49:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:35 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]: dispatch 2024-04-26T01:49:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:35 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]': finished 2024-04-26T01:49:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:35 smithi071 ceph-mon[32578]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T01:49:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:36.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:35 smithi037 ceph-mon[27906]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:36.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:35 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/999835689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]: dispatch 2024-04-26T01:49:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:35 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]: dispatch 2024-04-26T01:49:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:35 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bbe95809-3e82-4167-8b84-fed9410871e9"}]': finished 2024-04-26T01:49:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:35 smithi037 ceph-mon[27906]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T01:49:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:37.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:36 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/2808394526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:49:37.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:36 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/2808394526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:49:37.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:36 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/2808394526' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:49:38.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:37 smithi071 ceph-mon[32578]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:38.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:37 smithi037 ceph-mon[27906]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:38.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:37 smithi045 ceph-mon[32520]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:40.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:39 smithi071 ceph-mon[32578]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:40.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:39 smithi037 ceph-mon[27906]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:40.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:39 smithi045 ceph-mon[32520]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:42.112 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:41 smithi071 ceph-mon[32578]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:42.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:41 smithi037 ceph-mon[27906]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:42.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:41 smithi045 ceph-mon[32520]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:43.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:49:43.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:43.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:49:43.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:43.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T01:49:43.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:44.030 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:43 smithi071 ceph-mon[32578]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:44.031 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:43 smithi071 ceph-mon[32578]: Deploying daemon osd.5 on smithi071 2024-04-26T01:49:44.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:43 smithi037 ceph-mon[27906]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:44.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:43 smithi037 ceph-mon[27906]: Deploying daemon osd.5 on smithi071 2024-04-26T01:49:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:43 smithi045 ceph-mon[32520]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:43 smithi045 ceph-mon[32520]: Deploying daemon osd.5 on smithi071 2024-04-26T01:49:46.064 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:45 smithi071 ceph-mon[32578]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:46.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:45 smithi037 ceph-mon[27906]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:46.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:45 smithi045 ceph-mon[32520]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:47.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:47 smithi071 ceph-mon[32578]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:48.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:47 smithi037 ceph-mon[27906]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:48.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:47 smithi045 ceph-mon[32520]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:49.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:49.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:49.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:49.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:49.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:49.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:49.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:49.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:50 smithi071 ceph-mon[32578]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:50.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:50 smithi037 ceph-mon[27906]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:50.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:50 smithi045 ceph-mon[32520]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:51.202 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.204+0000 7f810c956700 1 -- 172.21.15.71:0/960341094 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f81081089c0 con 0x7f80f00a3cc0 2024-04-26T01:49:51.203 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 5 on host 'smithi071' 2024-04-26T01:49:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f80f00a3cc0 msgr2=0x7f80f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f80f00a3cc0 0x7f80f00a6180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f810806c090 tx=0x7f80f8006d20 comp rx=0 tx=0).stop 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 msgr2=0x7f810806bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f810806bb10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f80fc005f50 tx=0x7f80fc0047b0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 shutdown_connections 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f80f00a3cc0 0x7f80f00a6180 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8108103530 0x7f81081a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8108101990 0x7f810806bb10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 --2- 172.21.15.71:0/960341094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8108102b90 0x7f810819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 >> 172.21.15.71:0/960341094 conn(0x7f81080fcf30 msgr2=0x7f8108105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 shutdown_connections 2024-04-26T01:49:51.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:49:51.206+0000 7f810d958700 1 -- 172.21.15.71:0/960341094 wait complete. 2024-04-26T01:49:52.016 DEBUG:teuthology.orchestra.run.smithi071:osd.5> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T01:49:52.019 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-04-26T01:49:52.019 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:49:52.377 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:52.378 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.378 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.378 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.378 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.378 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: from='osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:49:52.378 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:52 smithi071 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: from='osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:49:52.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:52 smithi037 ceph-mon[27906]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:49:52.479 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:52.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: from='osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:49:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:52 smithi045 ceph-mon[32520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T01:49:53.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-mon[32578]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:53.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:49:53.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-mon[32578]: from='osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:49:53.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-mon[32578]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:49:53.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:53.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:49:53.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T01:49:53.753+0000 7fbd70d0c700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T01:49:53.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 01:49:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T01:49:53.759+0000 7fbd6caff700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:49:53.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:53 smithi037 ceph-mon[27906]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:53.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:53 smithi037 ceph-mon[27906]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:49:53.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:53 smithi037 ceph-mon[27906]: from='osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:49:53.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:53 smithi037 ceph-mon[27906]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:49:53.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:53.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:53 smithi037 ceph-mon[27906]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:49:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:53 smithi045 ceph-mon[32520]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:53 smithi045 ceph-mon[32520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T01:49:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:53 smithi045 ceph-mon[32520]: from='osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:49:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:53 smithi045 ceph-mon[32520]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T01:49:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:53 smithi045 ceph-mon[32520]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:49:54.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:54 smithi037 ceph-mon[27906]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:49:54.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:54 smithi037 ceph-mon[27906]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:49:54.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:54.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:54 smithi037 ceph-mon[27906]: osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] boot 2024-04-26T01:49:54.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:54 smithi037 ceph-mon[27906]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T01:49:54.884 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:54.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:54 smithi071 ceph-mon[32578]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:49:54.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:54 smithi071 ceph-mon[32578]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:49:54.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:54.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:54 smithi071 ceph-mon[32578]: osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] boot 2024-04-26T01:49:54.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:54 smithi071 ceph-mon[32578]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T01:49:54.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:54 smithi045 ceph-mon[32520]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:49:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:54 smithi045 ceph-mon[32520]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T01:49:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:54 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:54 smithi045 ceph-mon[32520]: osd.5 [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] boot 2024-04-26T01:49:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:54 smithi045 ceph-mon[32520]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T01:49:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:54 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:49:55.615 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:49:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:49:55.616 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T01:49:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T01:49:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.03751 s, 280 MB/s 2024-04-26T01:49:55.616 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T01:49:56.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:49:56.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi071 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:56.304 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:56.305 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:56.305 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.305 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:56 smithi071 ceph-mon[32578]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:49:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:49:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:49:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi071 2024-04-26T01:49:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:56.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:56.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:56.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.461 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:56 smithi037 ceph-mon[27906]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi071 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi071 to 8544M 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:49:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:56 smithi045 ceph-mon[32520]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T01:49:56.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-04-26T01:49:57.081 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:49:57.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:57 smithi037 ceph-mon[27906]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:49:57.568 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:57 smithi037 ceph-mon[27906]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:49:57.575 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:57 smithi071 ceph-mon[32578]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:49:57.576 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:57 smithi071 ceph-mon[32578]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:49:57.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:57 smithi045 ceph-mon[32520]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T01:49:57.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:57 smithi045 ceph-mon[32520]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:49:58.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:58 smithi037 ceph-mon[27906]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:49:58.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:58 smithi045 ceph-mon[32520]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:49:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:58 smithi071 ceph-mon[32578]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T01:49:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 -- 172.21.15.37:0/2396117392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c075420 msgr2=0x7f4c4c0758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 --2- 172.21.15.37:0/2396117392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c075420 0x7f4c4c0758a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009990 tx=0x7f4c34009ca0 comp rx=0 tx=0).stop 2024-04-26T01:49:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 -- 172.21.15.37:0/2396117392 shutdown_connections 2024-04-26T01:49:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 --2- 172.21.15.37:0/2396117392 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c10a9d0 0x7f4c4c10ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:59.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 --2- 172.21.15.37:0/2396117392 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c0769f0 0x7f4c4c074e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:59.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 --2- 172.21.15.37:0/2396117392 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c075420 0x7f4c4c0758a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:59.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 -- 172.21.15.37:0/2396117392 >> 172.21.15.37:0/2396117392 conn(0x7f4c4c0fe090 msgr2=0x7f4c4c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:49:59.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 -- 172.21.15.37:0/2396117392 shutdown_connections 2024-04-26T01:49:59.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.430+0000 7f4c51000700 1 -- 172.21.15.37:0/2396117392 wait complete. 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.431+0000 7f4c51000700 1 Processor -- start 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.431+0000 7f4c51000700 1 -- start start 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c51000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c075420 0x7f4c4c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c51000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c0769f0 0x7f4c4c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c51000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c10a9d0 0x7f4c4c1a2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c51000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c110730 con 0x7f4c4c0769f0 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c51000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c1105b0 con 0x7f4c4c075420 2024-04-26T01:49:59.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c51000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4c4c1108b0 con 0x7f4c4c10a9d0 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c41400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c0769f0 0x7f4c4c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c49e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c075420 0x7f4c4c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c41400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c0769f0 0x7f4c4c19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43232/0 (socket says 172.21.15.37:43232) 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c41400700 1 -- 172.21.15.37:0/3624145361 learned_addr learned my addr 172.21.15.37:0/3624145361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.432+0000 7f4c4a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c10a9d0 0x7f4c4c1a2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c49e00700 1 -- 172.21.15.37:0/3624145361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c10a9d0 msgr2=0x7f4c4c1a2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c49e00700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c10a9d0 0x7f4c4c1a2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c49e00700 1 -- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c0769f0 msgr2=0x7f4c4c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c49e00700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c0769f0 0x7f4c4c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c49e00700 1 -- 172.21.15.37:0/3624145361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c4c1a29b0 con 0x7f4c4c075420 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c4a800700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c10a9d0 0x7f4c4c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:49:59.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c49e00700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c075420 0x7f4c4c19ada0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00e9f0 tx=0x7f4c3c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:59.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c00c9c0 con 0x7f4c4c075420 2024-04-26T01:49:59.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3c00cb60 con 0x7f4c4c075420 2024-04-26T01:49:59.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c4c1a2ca0 con 0x7f4c4c075420 2024-04-26T01:49:59.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c018680 con 0x7f4c4c075420 2024-04-26T01:49:59.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.433+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c4c1a3190 con 0x7f4c4c075420 2024-04-26T01:49:59.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.435+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c4c069730 con 0x7f4c4c075420 2024-04-26T01:49:59.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.437+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4c3c010430 con 0x7f4c4c075420 2024-04-26T01:49:59.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.438+0000 7f4c43400700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4c300a3c40 0x7f4c300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:49:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.438+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4533+0+0 (secure 0 0 0) 0x7f4c3c0cf6a0 con 0x7f4c4c075420 2024-04-26T01:49:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.438+0000 7f4c41400700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4c300a3c40 0x7f4c300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:49:59.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.439+0000 7f4c41400700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4c300a3c40 0x7f4c300a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009750 tx=0x7f4c34005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:49:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.443+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c3c09cf40 con 0x7f4c4c075420 2024-04-26T01:49:59.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:49:59.685+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4c4c062e80 con 0x7f4c300a3c40 2024-04-26T01:49:59.683 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:59 smithi037 ceph-mon[27906]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-26T01:49:59.683 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:49:59 smithi037 ceph-mon[27906]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:49:59.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:59 smithi045 ceph-mon[32520]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-26T01:49:59.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:49:59 smithi045 ceph-mon[32520]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:49:59.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:59 smithi071 ceph-mon[32578]: pgmap v160: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 143 KiB/s, 0 objects/s recovering 2024-04-26T01:49:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:49:59 smithi071 ceph-mon[32578]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T01:50:00.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:00 smithi037 ceph-mon[27906]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:00 smithi037 ceph-mon[27906]: overall HEALTH_OK 2024-04-26T01:50:00.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:00 smithi045 ceph-mon[32520]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:00.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:00.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:00.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:00.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:00 smithi045 ceph-mon[32520]: overall HEALTH_OK 2024-04-26T01:50:00.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:00 smithi071 ceph-mon[32578]: from='client.24245 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:00.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:00.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:00.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:00.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:00 smithi071 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T01:50:01.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:01 smithi045 ceph-mon[32520]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T01:50:01.760 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:01 smithi037 ceph-mon[27906]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T01:50:01.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:01 smithi071 ceph-mon[32578]: pgmap v162: 1 pgs: 1 peering; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-04-26T01:50:03.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:03 smithi045 ceph-mon[32520]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:50:03.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:03 smithi071 ceph-mon[32578]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:50:03.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:03 smithi037 ceph-mon[27906]: pgmap v163: 1 pgs: 1 peering; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:50:04.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:04 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3055935024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af9e89d-771d-4ce4-8e7b-693cf99fe580"}]: dispatch 2024-04-26T01:50:04.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:04 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3055935024' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1af9e89d-771d-4ce4-8e7b-693cf99fe580"}]': finished 2024-04-26T01:50:04.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:04 smithi045 ceph-mon[32520]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T01:50:04.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:04.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:04 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3055935024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af9e89d-771d-4ce4-8e7b-693cf99fe580"}]: dispatch 2024-04-26T01:50:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:04 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3055935024' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1af9e89d-771d-4ce4-8e7b-693cf99fe580"}]': finished 2024-04-26T01:50:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:04 smithi071 ceph-mon[32578]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T01:50:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:04.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:04 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3055935024' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1af9e89d-771d-4ce4-8e7b-693cf99fe580"}]: dispatch 2024-04-26T01:50:04.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:04 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3055935024' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1af9e89d-771d-4ce4-8e7b-693cf99fe580"}]': finished 2024-04-26T01:50:04.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:04 smithi037 ceph-mon[27906]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T01:50:04.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:05 smithi071 ceph-mon[32578]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-26T01:50:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:05 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/922930668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:50:05.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:05 smithi037 ceph-mon[27906]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-26T01:50:05.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:05 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/922930668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:50:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:05 smithi045 ceph-mon[32520]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-04-26T01:50:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:05 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/922930668' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:50:07.807 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:07 smithi045 ceph-mon[32520]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:07.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:07 smithi071 ceph-mon[32578]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:07.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:07 smithi037 ceph-mon[27906]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:09.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:09 smithi037 ceph-mon[27906]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:09.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:09 smithi071 ceph-mon[32578]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:09 smithi045 ceph-mon[32520]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:11.779 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:11 smithi037 ceph-mon[27906]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:11.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:11 smithi071 ceph-mon[32578]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:12.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:11 smithi045 ceph-mon[32520]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:13.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:13 smithi037 ceph-mon[27906]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:13.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:13 smithi071 ceph-mon[32578]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:14.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:13 smithi045 ceph-mon[32520]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:14.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:14 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:50:14.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:14 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:14.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:50:14.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:14 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T01:50:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:14 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:15.695 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:15 smithi037 ceph-mon[27906]: Deploying daemon osd.6 on smithi037 2024-04-26T01:50:15.695 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:15 smithi037 ceph-mon[27906]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:15.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:15 smithi071 ceph-mon[32578]: Deploying daemon osd.6 on smithi037 2024-04-26T01:50:15.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:15 smithi071 ceph-mon[32578]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:16.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:15 smithi045 ceph-mon[32520]: Deploying daemon osd.6 on smithi037 2024-04-26T01:50:16.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:15 smithi045 ceph-mon[32520]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:17.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:17 smithi037 ceph-mon[27906]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:17 smithi071 ceph-mon[32578]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:18.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:17 smithi045 ceph-mon[32520]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:19.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:19 smithi071 ceph-mon[32578]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:19.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:19 smithi037 ceph-mon[27906]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:20.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:19 smithi045 ceph-mon[32520]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:21.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:21 smithi071 ceph-mon[32578]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:21.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:21 smithi037 ceph-mon[27906]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:22.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:21 smithi045 ceph-mon[32520]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:24.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:24 smithi037 ceph-mon[27906]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:24.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:24.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:24.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:24.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:24 smithi071 ceph-mon[32578]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:24.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:24 smithi045 ceph-mon[32520]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:26.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:26 smithi071 ceph-mon[32578]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:26.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:26 smithi037 ceph-mon[27906]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:26.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:26 smithi045 ceph-mon[32520]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:28.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:28 smithi037 ceph-mon[27906]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:28 smithi037 ceph-mon[27906]: from='osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:50:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:28 smithi037 ceph-mon[27906]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:50:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:28 smithi071 ceph-mon[32578]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:28 smithi071 ceph-mon[32578]: from='osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:50:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:28 smithi071 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:50:28.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.496+0000 7f4c43400700 1 -- 172.21.15.37:0/3624145361 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c4c062e80 con 0x7f4c300a3c40 2024-04-26T01:50:28.496 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4c300a3c40 msgr2=0x7f4c300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4c300a3c40 0x7f4c300a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4c34009750 tx=0x7f4c34005960 comp rx=0 tx=0).stop 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c075420 msgr2=0x7f4c4c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c075420 0x7f4c4c19ada0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c00e9f0 tx=0x7f4c3c00ed00 comp rx=0 tx=0).stop 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 shutdown_connections 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4c300a3c40 0x7f4c300a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4c4c10a9d0 0x7f4c4c1a2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4c4c075420 0x7f4c4c19ada0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 --2- 172.21.15.37:0/3624145361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4c4c0769f0 0x7f4c4c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:28.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 >> 172.21.15.37:0/3624145361 conn(0x7f4c4c0fe090 msgr2=0x7f4c4c106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:28.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 shutdown_connections 2024-04-26T01:50:28.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:50:28.500+0000 7f4c51000700 1 -- 172.21.15.37:0/3624145361 wait complete. 2024-04-26T01:50:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:28 smithi045 ceph-mon[32520]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:28.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:28 smithi045 ceph-mon[32520]: from='osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:50:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:28 smithi045 ceph-mon[32520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T01:50:29.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:50:29.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:50:29.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:29.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:50:29.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:50:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:29 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:29.325 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T01:50:29.331 INFO:tasks.cephadm:Deploying osd.7 on smithi045 with /dev/vg_nvme/lv_2... 2024-04-26T01:50:29.331 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:50:29.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:29 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:29.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:29 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:29.587 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:50:30.180 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-mon[27906]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:30.181 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-mon[27906]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:50:30.181 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-mon[27906]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:50:30.181 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:30.181 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:30.181 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T01:50:30.136+0000 7ff8a8000700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T01:50:30.181 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 01:50:30 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T01:50:30.181+0000 7ff89de00700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:50:30.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:30 smithi045 ceph-mon[32520]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:30.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:30 smithi045 ceph-mon[32520]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:50:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:30 smithi045 ceph-mon[32520]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:50:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:30 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:30 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:30.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:30 smithi071 ceph-mon[32578]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:30 smithi071 ceph-mon[32578]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:50:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:30 smithi071 ceph-mon[32578]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T01:50:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:30 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:30 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:31.354 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:50:31.354 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:50:31.354 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-26T01:50:31.354 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-26T01:50:31.354 INFO:teuthology.orchestra.run.smithi045.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00817412 s, 1.3 GB/s 2024-04-26T01:50:31.354 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-26T01:50:31.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:31 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:50:31.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:31 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:50:31.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:31 smithi037 ceph-mon[27906]: from='osd.6 ' entity='osd.6' 2024-04-26T01:50:31.359 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:31.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:31 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:50:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:31 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:50:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:31 smithi071 ceph-mon[32578]: from='osd.6 ' entity='osd.6' 2024-04-26T01:50:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:31.406 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:31 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:50:31.406 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:31 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:50:31.406 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:31 smithi045 ceph-mon[32520]: from='osd.6 ' entity='osd.6' 2024-04-26T01:50:31.406 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:31.985 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_2 2024-04-26T01:50:32.232 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:50:32.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:32 smithi045 ceph-mon[32520]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:32 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:32 smithi045 ceph-mon[32520]: osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] boot 2024-04-26T01:50:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:32 smithi045 ceph-mon[32520]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T01:50:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:32 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:32.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:32 smithi071 ceph-mon[32578]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:32 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:32 smithi071 ceph-mon[32578]: osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] boot 2024-04-26T01:50:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:32 smithi071 ceph-mon[32578]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T01:50:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:32 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:32 smithi037 ceph-mon[27906]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-26T01:50:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:32 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:32 smithi037 ceph-mon[27906]: osd.6 [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] boot 2024-04-26T01:50:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:32 smithi037 ceph-mon[27906]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T01:50:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:32 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:50:33.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:33 smithi037 ceph-mon[27906]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:50:33.489 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:33 smithi045 ceph-mon[32520]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:50:33.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:33 smithi071 ceph-mon[32578]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1678686928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4102ba0 msgr2=0x7fdfd4103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1678686928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4102ba0 0x7fdfd4103000 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdfbc0099c0 tx=0x7fdfbc009cd0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1678686928 shutdown_connections 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1678686928 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd4103540 0x7fdfd4109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1678686928 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd41019a0 0x7fdfd4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1678686928 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4102ba0 0x7fdfd4103000 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.801 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1678686928 >> 172.21.15.45:0/1678686928 conn(0x7fdfd40fcf40 msgr2=0x7fdfd40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1678686928 shutdown_connections 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1678686928 wait complete. 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.801+0000 7fdfd96f8700 1 Processor -- start 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 -- start start 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd41019a0 0x7fdfd406b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd4102ba0 0x7fdfd419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 0x7fdfd41a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410d4b0 con 0x7fdfd41019a0 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410d330 con 0x7fdfd4103540 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd96f8700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd410d630 con 0x7fdfd4102ba0 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 0x7fdfd41a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 0x7fdfd41a0050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:54250/0 (socket says 172.21.15.45:54250) 2024-04-26T01:50:33.802 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 -- 172.21.15.45:0/1207741837 learned_addr learned my addr 172.21.15.45:0/1207741837 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd27fc700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd4102ba0 0x7fdfd419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 -- 172.21.15.45:0/1207741837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd4102ba0 msgr2=0x7fdfd419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd4102ba0 0x7fdfd419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 -- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd41019a0 msgr2=0x7fdfd406b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd41019a0 0x7fdfd406b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 -- 172.21.15.45:0/1207741837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd41a0920 con 0x7fdfd4103540 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.802+0000 7fdfd37fe700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 0x7fdfd41a0050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00b550 tx=0x7fdfcc00b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.803+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc015020 con 0x7fdfd4103540 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.803+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41a0c10 con 0x7fdfd4103540 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.803+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfcc011ce0 con 0x7fdfd4103540 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.803+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc010d40 con 0x7fdfd4103540 2024-04-26T01:50:33.803 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.803+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd41a1150 con 0x7fdfd4103540 2024-04-26T01:50:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.804+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4069730 con 0x7fdfd4103540 2024-04-26T01:50:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.805+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfcc018020 con 0x7fdfd4103540 2024-04-26T01:50:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.805+0000 7fdfcbfff700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdfc00a3c40 0x7fdfc00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.806+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7fdfcc0ceec0 con 0x7fdfd4103540 2024-04-26T01:50:33.804 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.806+0000 7fdfd2ffd700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdfc00a3c40 0x7fdfc00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:33.805 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.807+0000 7fdfd2ffd700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdfc00a3c40 0x7fdfc00a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4102a00 tx=0x7fdfc4008560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:33.808 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:33.810+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfcc01f020 con 0x7fdfd4103540 2024-04-26T01:50:34.024 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:34.026+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fdfd4062e80 con 0x7fdfc00a3c40 2024-04-26T01:50:34.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:34 smithi037 ceph-mon[27906]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:34.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:34 smithi045 ceph-mon[32520]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:34.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:34 smithi071 ceph-mon[32578]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:34.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:34.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:34.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:35.371 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:35.372 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:35 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:35.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:35.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:35 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:50:35.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:35.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:35.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:35.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:35.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:35 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:36 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi037 2024-04-26T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:36 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi037 to 5698M 2024-04-26T01:50:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:36 smithi045 ceph-mon[32520]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:36.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:36 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi037 2024-04-26T01:50:36.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:36 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi037 to 5698M 2024-04-26T01:50:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:36 smithi037 ceph-mon[27906]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:36.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:36 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi037 2024-04-26T01:50:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:36 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi037 to 5698M 2024-04-26T01:50:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:36 smithi071 ceph-mon[32578]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:37.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:37 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/1905792628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]: dispatch 2024-04-26T01:50:37.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:37 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]: dispatch 2024-04-26T01:50:37.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:37 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]': finished 2024-04-26T01:50:37.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:37 smithi037 ceph-mon[27906]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T01:50:37.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:37 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:37.470 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:37 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/1905792628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]: dispatch 2024-04-26T01:50:37.470 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:37 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]: dispatch 2024-04-26T01:50:37.471 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:37 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]': finished 2024-04-26T01:50:37.471 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:37 smithi071 ceph-mon[32578]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T01:50:37.471 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:37 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:37.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:37 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/1905792628' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]: dispatch 2024-04-26T01:50:37.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:37 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]: dispatch 2024-04-26T01:50:37.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:37 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3389b007-2053-4508-a620-d820a3a6fecf"}]': finished 2024-04-26T01:50:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:37 smithi045 ceph-mon[32520]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T01:50:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:37 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:38.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:38 smithi037 ceph-mon[27906]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:38 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/3093974057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:50:38.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:38 smithi045 ceph-mon[32520]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:38 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/3093974057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:50:38.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:38 smithi071 ceph-mon[32578]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:38.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:38 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/3093974057' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:50:40.456 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:40 smithi045 ceph-mon[32520]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:40.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:40 smithi037 ceph-mon[27906]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:40.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:40 smithi071 ceph-mon[32578]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:42.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:42 smithi037 ceph-mon[27906]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:42 smithi045 ceph-mon[32520]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:42.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:42 smithi071 ceph-mon[32578]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:44.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:44 smithi037 ceph-mon[27906]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:50:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:44.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:44 smithi045 ceph-mon[32520]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:44.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:50:44.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:44.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:44 smithi071 ceph-mon[32578]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T01:50:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:45.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:45 smithi045 ceph-mon[32520]: Deploying daemon osd.7 on smithi045 2024-04-26T01:50:45.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:45 smithi071 ceph-mon[32578]: Deploying daemon osd.7 on smithi045 2024-04-26T01:50:45.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:45 smithi037 ceph-mon[27906]: Deploying daemon osd.7 on smithi045 2024-04-26T01:50:46.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:46 smithi045 ceph-mon[32520]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:46.609 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:46 smithi037 ceph-mon[27906]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:46.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:46 smithi071 ceph-mon[32578]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:48.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:48 smithi037 ceph-mon[27906]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:48 smithi045 ceph-mon[32520]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:48.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:48 smithi071 ceph-mon[32578]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:50.438 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:50 smithi045 ceph-mon[32520]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:50.438 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:50.438 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:50.438 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:50.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:50 smithi071 ceph-mon[32578]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:50.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:50.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:50.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:50 smithi037 ceph-mon[27906]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:52 smithi045 ceph-mon[32520]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:52.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:52 smithi071 ceph-mon[32578]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:52.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:52 smithi037 ceph-mon[27906]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:53.009 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.010+0000 7fdfcbfff700 1 -- 172.21.15.45:0/1207741837 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdfd4062e80 con 0x7fdfc00a3c40 2024-04-26T01:50:53.009 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 7 on host 'smithi045' 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdfc00a3c40 msgr2=0x7fdfc00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdfc00a3c40 0x7fdfc00a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4102a00 tx=0x7fdfc4008560 comp rx=0 tx=0).stop 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 msgr2=0x7fdfd41a0050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 0x7fdfd41a0050 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00b550 tx=0x7fdfcc00b910 comp rx=0 tx=0).stop 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 shutdown_connections 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfd41019a0 0x7fdfd406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fdfc00a3c40 0x7fdfc00a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdfd4103540 0x7fdfd41a0050 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 --2- 172.21.15.45:0/1207741837 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdfd4102ba0 0x7fdfd419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:53.013 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.013+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 >> 172.21.15.45:0/1207741837 conn(0x7fdfd40fcf40 msgr2=0x7fdfd4105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:53.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.014+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 shutdown_connections 2024-04-26T01:50:53.014 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:50:53.014+0000 7fdfd96f8700 1 -- 172.21.15.45:0/1207741837 wait complete. 2024-04-26T01:50:53.734 DEBUG:teuthology.orchestra.run.smithi045:osd.7> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T01:50:53.736 INFO:tasks.cephadm:Deploying osd.8 on smithi071 with /dev/vg_nvme/lv_2... 2024-04-26T01:50:53.737 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T01:50:53.980 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:50:54.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:53 smithi045 ceph-mon[32520]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:54.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:53 smithi045 ceph-mon[32520]: from='osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:50:54.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:53 smithi045 ceph-mon[32520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:50:54.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:53 smithi071 ceph-mon[32578]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:54.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:53 smithi071 ceph-mon[32578]: from='osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:50:54.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:53 smithi071 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:50:54.102 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:53 smithi037 ceph-mon[27906]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:53 smithi037 ceph-mon[27906]: from='osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:50:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:53 smithi037 ceph-mon[27906]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T01:50:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:55.218 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:55.218 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:54 smithi071 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:50:55.219 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:54 smithi071 ceph-mon[32578]: from='osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:50:55.219 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:54 smithi071 ceph-mon[32578]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:50:55.219 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:55.219 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:54 smithi071 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:50:55.250 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:54 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:55.250 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:54 smithi045 ceph-mon[32520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:50:55.250 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:54 smithi045 ceph-mon[32520]: from='osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:50:55.250 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:54 smithi045 ceph-mon[32520]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:50:55.250 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:54 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:55.251 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:54 smithi045 ceph-mon[32520]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:50:55.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:55.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:54 smithi037 ceph-mon[27906]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T01:50:55.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:54 smithi037 ceph-mon[27906]: from='osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:50:55.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:54 smithi037 ceph-mon[27906]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T01:50:55.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:55.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:54 smithi037 ceph-mon[27906]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:50:55.504 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T01:50:55.250+0000 7f0538065700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T01:50:55.504 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T01:50:55.265+0000 7f0533e58700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:50:55.882 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T01:50:55.882 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T01:50:55.882 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-26T01:50:55.882 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-26T01:50:55.882 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0139486 s, 752 MB/s 2024-04-26T01:50:55.882 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-26T01:50:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:55 smithi071 ceph-mon[32578]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:55 smithi071 ceph-mon[32578]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:50:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:55 smithi071 ceph-mon[32578]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:50:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:56.190 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-mon[32520]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:56.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-mon[32520]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:50:56.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-mon[32520]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:50:56.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:56.191 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:56.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:55 smithi037 ceph-mon[27906]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:56.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:55 smithi037 ceph-mon[27906]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:50:56.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:55 smithi037 ceph-mon[27906]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T01:50:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:56.488 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_2 2024-04-26T01:50:56.731 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:50:57.065 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:56 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:50:57.065 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:56 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:50:57.065 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:57.065 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:56 smithi071 ceph-mon[32578]: osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] boot 2024-04-26T01:50:57.065 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:56 smithi071 ceph-mon[32578]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T01:50:57.065 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:56 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:50:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:56 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:50:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:56 smithi045 ceph-mon[32520]: osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] boot 2024-04-26T01:50:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:56 smithi045 ceph-mon[32520]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T01:50:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:57.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:56 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:50:57.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:56 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:50:57.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:57.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:56 smithi037 ceph-mon[27906]: osd.7 [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] boot 2024-04-26T01:50:57.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:56 smithi037 ceph-mon[27906]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T01:50:57.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:50:58.087 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:58.087 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:50:58.087 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.087 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.087 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.087 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:58.088 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:50:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:58.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 -- 172.21.15.71:0/509950858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034103ca0 msgr2=0x7f4034104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 --2- 172.21.15.71:0/509950858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034103ca0 0x7f4034104100 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4024009990 tx=0x7f4024009ca0 comp rx=0 tx=0).stop 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 -- 172.21.15.71:0/509950858 shutdown_connections 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 --2- 172.21.15.71:0/509950858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034102aa0 0x7f4034102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 --2- 172.21.15.71:0/509950858 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4034104640 0x7f403410aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 --2- 172.21.15.71:0/509950858 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034103ca0 0x7f4034104100 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 -- 172.21.15.71:0/509950858 >> 172.21.15.71:0/509950858 conn(0x7f40340fe040 msgr2=0x7f4034100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:50:58.311 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.312+0000 7f4039dda700 1 -- 172.21.15.71:0/509950858 shutdown_connections 2024-04-26T01:50:58.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.313+0000 7f4039dda700 1 -- 172.21.15.71:0/509950858 wait complete. 2024-04-26T01:50:58.312 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.313+0000 7f4039dda700 1 Processor -- start 2024-04-26T01:50:58.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.313+0000 7f4039dda700 1 -- start start 2024-04-26T01:50:58.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034102aa0 0x7f4034198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:58.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4034103ca0 0x7f40341991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:58.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 0x7f40341a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f403410e5d0 con 0x7f4034102aa0 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f403410e450 con 0x7f4034103ca0 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f403410e750 con 0x7f4034104640 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 0x7f40341a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 0x7f40341a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:48398/0 (socket says 172.21.15.71:48398) 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 -- 172.21.15.71:0/1684162177 learned_addr learned my addr 172.21.15.71:0/1684162177 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4032ffd700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4034103ca0 0x7f40341991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 -- 172.21.15.71:0/1684162177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4034103ca0 msgr2=0x7f40341991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4034103ca0 0x7f40341991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 -- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034102aa0 msgr2=0x7f4034198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034102aa0 0x7f4034198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 -- 172.21.15.71:0/1684162177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40341a08f0 con 0x7f4034104640 2024-04-26T01:50:58.314 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4033fff700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 0x7f40341a0220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f402c007ee0 tx=0x7f402c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:58.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c00dbd0 con 0x7f4034104640 2024-04-26T01:50:58.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40341a0be0 con 0x7f4034104640 2024-04-26T01:50:58.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f402c00dd70 con 0x7f4034104640 2024-04-26T01:50:58.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c005990 con 0x7f4034104640 2024-04-26T01:50:58.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.314+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40341a1170 con 0x7f4034104640 2024-04-26T01:50:58.315 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.315+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4034069730 con 0x7f4034104640 2024-04-26T01:50:58.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.317+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f402c01a020 con 0x7f4034104640 2024-04-26T01:50:58.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.317+0000 7f4030ff9700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40200a3c40 0x7f40200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:50:58.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.317+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f402c0cf730 con 0x7f4034104640 2024-04-26T01:50:58.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.318+0000 7f40337fe700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40200a3c40 0x7f40200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:50:58.316 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.318+0000 7f40337fe700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40200a3c40 0x7f40200a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4034199b50 tx=0x7f401c006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:50:58.318 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.321+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f402c010070 con 0x7f4034104640 2024-04-26T01:50:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-04-26T01:50:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T01:50:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:50:58.527 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:50:58.529+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4034062e80 con 0x7f40200a3c40 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi045 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi045 to 5697M 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi045 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi045 to 5697M 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:59.282 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi045 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi045 to 5697M 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:50:59.427 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:59 smithi045 ceph-mon[32520]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:59 smithi045 ceph-mon[32520]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:50:59 smithi045 ceph-mon[32520]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:51:00.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:59 smithi071 ceph-mon[32578]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:00.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:59 smithi071 ceph-mon[32578]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:00.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:50:59 smithi071 ceph-mon[32578]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:51:00.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:59 smithi037 ceph-mon[27906]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:00.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:59 smithi037 ceph-mon[27906]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:50:59 smithi037 ceph-mon[27906]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T01:51:01.624 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:01 smithi071 ceph-mon[32578]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:51:01.625 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:01 smithi071 ceph-mon[32578]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:01.625 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:01 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/3212651181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]: dispatch 2024-04-26T01:51:01.625 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:01 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]: dispatch 2024-04-26T01:51:01.625 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:01 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]': finished 2024-04-26T01:51:01.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:01 smithi037 ceph-mon[27906]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:51:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:01 smithi037 ceph-mon[27906]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:01 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/3212651181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]: dispatch 2024-04-26T01:51:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:01 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]: dispatch 2024-04-26T01:51:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:01 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]': finished 2024-04-26T01:51:01.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:01 smithi045 ceph-mon[32520]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T01:51:01.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:01 smithi045 ceph-mon[32520]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:01.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:01 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/3212651181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]: dispatch 2024-04-26T01:51:01.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:01 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]: dispatch 2024-04-26T01:51:01.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:01 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ad43bcc8-3414-401c-bbff-6ff4ed6a3149"}]': finished 2024-04-26T01:51:02.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:02 smithi071 ceph-mon[32578]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T01:51:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:02 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:02 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/276869004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:51:02.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:02 smithi037 ceph-mon[27906]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T01:51:02.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:02 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:02.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:02 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/276869004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:51:02.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:02 smithi045 ceph-mon[32520]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T01:51:02.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:02 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:02.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:02 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/276869004' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:51:03.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:03 smithi071 ceph-mon[32578]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:03 smithi037 ceph-mon[27906]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:03.753 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:03 smithi045 ceph-mon[32520]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 234 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:05.752 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:05 smithi071 ceph-mon[32578]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:05.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:05 smithi037 ceph-mon[27906]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:05 smithi045 ceph-mon[32520]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:07.786 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:07 smithi071 ceph-mon[32578]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:07.806 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:07 smithi045 ceph-mon[32520]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:07.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:07 smithi037 ceph-mon[27906]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:09.626 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:09 smithi071 ceph-mon[32578]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:09.626 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:51:09.626 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:09.626 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:09 smithi071 ceph-mon[32578]: Deploying daemon osd.8 on smithi071 2024-04-26T01:51:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:09 smithi037 ceph-mon[27906]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:51:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:09 smithi037 ceph-mon[27906]: Deploying daemon osd.8 on smithi071 2024-04-26T01:51:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:09 smithi045 ceph-mon[32520]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T01:51:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:09 smithi045 ceph-mon[32520]: Deploying daemon osd.8 on smithi071 2024-04-26T01:51:11.824 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:11 smithi071 ceph-mon[32578]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:11.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:11 smithi037 ceph-mon[27906]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:12.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:11 smithi045 ceph-mon[32520]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:13.551 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:13 smithi071 ceph-mon[32578]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:13.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:13 smithi037 ceph-mon[27906]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:14.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:13 smithi045 ceph-mon[32520]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:14.791 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:14 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:14.791 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:14 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:14.791 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:14 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:14.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:14.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:14.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:14 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:14 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:14 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:14 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:16.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:16 smithi071 ceph-mon[32578]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:16.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:16 smithi037 ceph-mon[27906]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:16 smithi045 ceph-mon[32520]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:18.015 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.016+0000 7f4030ff9700 1 -- 172.21.15.71:0/1684162177 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4034062e80 con 0x7f40200a3c40 2024-04-26T01:51:18.015 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 8 on host 'smithi071' 2024-04-26T01:51:18.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40200a3c40 msgr2=0x7f40200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:18.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40200a3c40 0x7f40200a6100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4034199b50 tx=0x7f401c006d20 comp rx=0 tx=0).stop 2024-04-26T01:51:18.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 msgr2=0x7f40341a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:18.017 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 0x7f40341a0220 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f402c007ee0 tx=0x7f402c00d3b0 comp rx=0 tx=0).stop 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 shutdown_connections 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4034102aa0 0x7f4034198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40200a3c40 0x7f40200a6100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4034104640 0x7f40341a0220 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 --2- 172.21.15.71:0/1684162177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4034103ca0 0x7f40341991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 >> 172.21.15.71:0/1684162177 conn(0x7f40340fe040 msgr2=0x7f40341070c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 shutdown_connections 2024-04-26T01:51:18.018 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:51:18.018+0000 7f4039dda700 1 -- 172.21.15.71:0/1684162177 wait complete. 2024-04-26T01:51:18.331 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:18.331 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.331 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.331 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:18.331 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:18.332 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.332 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:18.332 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.332 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.332 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:51:18.332 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:18 smithi071 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:51:18.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:18 smithi037 ceph-mon[27906]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:18.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:18.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:51:18.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:18 smithi045 ceph-mon[32520]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T01:51:18.651 DEBUG:teuthology.orchestra.run.smithi071:osd.8> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T01:51:18.653 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-04-26T01:51:18.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:51:19.120 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:20.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-mon[32578]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:20.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:51:20.028 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-mon[32578]: from='osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:51:20.029 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-mon[32578]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:51:20.029 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:20.029 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:51:20.029 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T01:51:19.974+0000 7f7be1ac0700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T01:51:20.029 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 01:51:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T01:51:19.985+0000 7f7bdd8b3700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:51:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:19 smithi045 ceph-mon[32520]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:19 smithi045 ceph-mon[32520]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:51:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:19 smithi045 ceph-mon[32520]: from='osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:51:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:19 smithi045 ceph-mon[32520]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:51:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:19 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:19 smithi045 ceph-mon[32520]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:51:20.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:19 smithi037 ceph-mon[27906]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:20.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:19 smithi037 ceph-mon[27906]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T01:51:20.300 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:19 smithi037 ceph-mon[27906]: from='osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:51:20.300 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:19 smithi037 ceph-mon[27906]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T01:51:20.300 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:19 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:20.300 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:19 smithi037 ceph-mon[27906]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:51:21.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:20 smithi037 ceph-mon[27906]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:51:21.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:20 smithi037 ceph-mon[27906]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:51:21.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:20 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:21.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:20 smithi037 ceph-mon[27906]: osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] boot 2024-04-26T01:51:21.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:20 smithi037 ceph-mon[27906]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T01:51:21.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:20 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:21.128 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:20 smithi071 ceph-mon[32578]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:51:21.128 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:20 smithi071 ceph-mon[32578]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:51:21.128 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:20 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:21.129 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:20 smithi071 ceph-mon[32578]: osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] boot 2024-04-26T01:51:21.129 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:20 smithi071 ceph-mon[32578]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T01:51:21.129 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:20 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:20 smithi045 ceph-mon[32520]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:51:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:20 smithi045 ceph-mon[32520]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T01:51:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:20 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:20 smithi045 ceph-mon[32520]: osd.8 [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] boot 2024-04-26T01:51:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:20 smithi045 ceph-mon[32520]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T01:51:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:20 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:51:22.038 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:51:22.038 INFO:teuthology.orchestra.run.smithi037.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:51:22.038 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 10+0 records in 2024-04-26T01:51:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:10+0 records out 2024-04-26T01:51:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0351039 s, 299 MB/s 2024-04-26T01:51:22.039 INFO:teuthology.orchestra.run.smithi037.stderr: stderr: 2024-04-26T01:51:22.039 INFO:teuthology.orchestra.run.smithi037.stderr:--> Zapping successful for: 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:22.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:21 smithi045 ceph-mon[32520]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:22.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:22.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:21 smithi071 ceph-mon[32578]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:51:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:51:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:51:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-04-26T01:51:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:22.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:21 smithi037 ceph-mon[27906]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T01:51:22.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-04-26T01:51:23.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:23 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi071 2024-04-26T01:51:23.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:23 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-26T01:51:23.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:23 smithi037 ceph-mon[27906]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:51:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:23 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi071 2024-04-26T01:51:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:23 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-26T01:51:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:23 smithi045 ceph-mon[32520]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:51:23.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:23 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi071 2024-04-26T01:51:23.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:23 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi071 to 5696M 2024-04-26T01:51:23.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:23 smithi071 ceph-mon[32578]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T01:51:23.427 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:51:24.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:24 smithi037 ceph-mon[27906]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:24.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:24 smithi071 ceph-mon[32578]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:24 smithi045 ceph-mon[32520]: pgmap v221: 1 pgs: 1 remapped+peering; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:25.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:25 smithi037 ceph-mon[27906]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:51:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:25 smithi071 ceph-mon[32578]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:51:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:25 smithi045 ceph-mon[32520]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T01:51:25.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 -- 172.21.15.37:0/3209842663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 msgr2=0x7fdea0075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 --2- 172.21.15.37:0/3209842663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 0x7fdea0075920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fde8c009990 tx=0x7fde8c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 -- 172.21.15.37:0/3209842663 shutdown_connections 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 --2- 172.21.15.37:0/3209842663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdea0076a70 0x7fdea0074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 --2- 172.21.15.37:0/3209842663 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea010f070 0x7fdea0111460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 --2- 172.21.15.37:0/3209842663 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 0x7fdea0075920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.806+0000 7fdea5400700 1 -- 172.21.15.37:0/3209842663 >> 172.21.15.37:0/3209842663 conn(0x7fdea00fe030 msgr2=0x7fdea0100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.807+0000 7fdea5400700 1 -- 172.21.15.37:0/3209842663 shutdown_connections 2024-04-26T01:51:25.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.807+0000 7fdea5400700 1 -- 172.21.15.37:0/3209842663 wait complete. 2024-04-26T01:51:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.808+0000 7fdea5400700 1 Processor -- start 2024-04-26T01:51:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.808+0000 7fdea5400700 1 -- start start 2024-04-26T01:51:25.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.808+0000 7fdea5400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 0x7fdea019f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fdea5400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 0x7fdea019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fdea5400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdea010f070 0x7fdea01a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fdea5400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0114b70 con 0x7fdea010f070 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fdea5400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdea01149f0 con 0x7fdea0076a70 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fdea5400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdea0114cf0 con 0x7fdea00754a0 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 0x7fdea019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 0x7fdea019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:46932/0 (socket says 172.21.15.37:46932) 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9f200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdea010f070 0x7fdea01a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 -- 172.21.15.37:0/2180609094 learned_addr learned my addr 172.21.15.37:0/2180609094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:51:25.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9e800700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 0x7fdea019f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 -- 172.21.15.37:0/2180609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 msgr2=0x7fdea019f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 0x7fdea019f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 -- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdea010f070 msgr2=0x7fdea01a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdea010f070 0x7fdea01a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.809+0000 7fde9de00700 1 -- 172.21.15.37:0/2180609094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea01a6f20 con 0x7fdea0076a70 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.810+0000 7fde9de00700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 0x7fdea019f800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0047b0 tx=0x7fde8c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.810+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c026070 con 0x7fdea0076a70 2024-04-26T01:51:25.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.810+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c021410 con 0x7fdea0076a70 2024-04-26T01:51:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.810+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea01a71b0 con 0x7fdea0076a70 2024-04-26T01:51:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.810+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c017650 con 0x7fdea0076a70 2024-04-26T01:51:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.810+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea01a7600 con 0x7fdea0076a70 2024-04-26T01:51:25.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.812+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea0069730 con 0x7fdea0076a70 2024-04-26T01:51:25.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.813+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde8c029070 con 0x7fdea0076a70 2024-04-26T01:51:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.815+0000 7fde97400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde840a3c40 0x7fde840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.815+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5829+0+0 (secure 0 0 0) 0x7fde8c0d0920 con 0x7fdea0076a70 2024-04-26T01:51:25.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.815+0000 7fde9e800700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde840a3c40 0x7fde840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:25.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.816+0000 7fde9e800700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde840a3c40 0x7fde840a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdea0076e50 tx=0x7fde90009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:25.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:25.819+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde8c05dff0 con 0x7fdea0076a70 2024-04-26T01:51:26.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:26.064+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdea0062f40 con 0x7fde840a3c40 2024-04-26T01:51:26.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:26 smithi037 ceph-mon[27906]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:26.063 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:26 smithi037 ceph-mon[27906]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:51:26.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:26 smithi071 ceph-mon[32578]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:26.303 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:26 smithi071 ceph-mon[32578]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:51:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:26 smithi045 ceph-mon[32520]: pgmap v224: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:26 smithi045 ceph-mon[32520]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T01:51:27.347 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:27 smithi037 ceph-mon[27906]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:27.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:51:27.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:51:27.348 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:27.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:27 smithi071 ceph-mon[32578]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:27.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:51:27.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:51:27.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:27.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:27 smithi045 ceph-mon[32520]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:51:27.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:51:27.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:51:27.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:28.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:28 smithi037 ceph-mon[27906]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:28 smithi071 ceph-mon[32578]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:28 smithi045 ceph-mon[32520]: pgmap v226: 1 pgs: 1 remapped+peering; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:30.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:30 smithi071 ceph-mon[32578]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:51:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:30 smithi037 ceph-mon[27906]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:51:30.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:30 smithi045 ceph-mon[32520]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:51:31.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:31 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1860234586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]: dispatch 2024-04-26T01:51:31.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:31 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]: dispatch 2024-04-26T01:51:31.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:31 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]': finished 2024-04-26T01:51:31.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:31 smithi037 ceph-mon[27906]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T01:51:31.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:31.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:31 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1860234586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]: dispatch 2024-04-26T01:51:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:31 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]: dispatch 2024-04-26T01:51:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:31 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]': finished 2024-04-26T01:51:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:31 smithi071 ceph-mon[32578]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T01:51:31.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:31 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1860234586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]: dispatch 2024-04-26T01:51:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:31 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]: dispatch 2024-04-26T01:51:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:31 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22321723-8f84-485f-9ecc-fe50bf7c0233"}]': finished 2024-04-26T01:51:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:31 smithi045 ceph-mon[32520]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T01:51:31.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:32 smithi071 ceph-mon[32578]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T01:51:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:32 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1167839333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:51:32.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:32 smithi037 ceph-mon[27906]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T01:51:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:32 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1167839333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:51:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:32 smithi045 ceph-mon[32520]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-04-26T01:51:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:32 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1167839333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:51:34.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:34 smithi071 ceph-mon[32578]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:51:34.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:34 smithi037 ceph-mon[27906]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:51:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:34 smithi045 ceph-mon[32520]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:51:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:36 smithi071 ceph-mon[32578]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T01:51:36.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:36 smithi037 ceph-mon[27906]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T01:51:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:36 smithi045 ceph-mon[32520]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-04-26T01:51:38.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:38 smithi037 ceph-mon[27906]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:51:38.306 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:38 smithi045 ceph-mon[32520]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:51:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:38 smithi071 ceph-mon[32578]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:51:40.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:40 smithi037 ceph-mon[27906]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:40 smithi071 ceph-mon[32578]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:40.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:40 smithi045 ceph-mon[32520]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:41.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:51:41.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:41.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:51:41.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:41.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T01:51:41.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:51:42.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:42 smithi071 ceph-mon[32578]: Deploying daemon osd.9 on smithi037 2024-04-26T01:51:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:42 smithi071 ceph-mon[32578]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:42 smithi037 ceph-mon[27906]: Deploying daemon osd.9 on smithi037 2024-04-26T01:51:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:42 smithi037 ceph-mon[27906]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:42.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:42 smithi045 ceph-mon[32520]: Deploying daemon osd.9 on smithi037 2024-04-26T01:51:42.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:42 smithi045 ceph-mon[32520]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:44.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:44 smithi037 ceph-mon[27906]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:44.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:44 smithi071 ceph-mon[32578]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:44.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:44 smithi045 ceph-mon[32520]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:46.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:46 smithi037 ceph-mon[27906]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:46 smithi071 ceph-mon[32578]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:46 smithi045 ceph-mon[32520]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:48 smithi071 ceph-mon[32578]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:48.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:48 smithi037 ceph-mon[27906]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:48.503 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:48 smithi045 ceph-mon[32520]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:50.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:50 smithi037 ceph-mon[27906]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:50.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:50.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:50.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:50 smithi071 ceph-mon[32578]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:50 smithi045 ceph-mon[32520]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:51:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:52.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:52 smithi037 ceph-mon[27906]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:52 smithi071 ceph-mon[32578]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:52 smithi045 ceph-mon[32520]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:54.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.077+0000 7fde97400700 1 -- 172.21.15.37:0/2180609094 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdea0062f40 con 0x7fde840a3c40 2024-04-26T01:51:54.077 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.080+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde840a3c40 msgr2=0x7fde840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.080+0000 7fdea5400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde840a3c40 0x7fde840a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdea0076e50 tx=0x7fde90009260 comp rx=0 tx=0).stop 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.080+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 msgr2=0x7fdea019f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.080+0000 7fdea5400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 0x7fdea019f800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fde8c0047b0 tx=0x7fde8c009510 comp rx=0 tx=0).stop 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 shutdown_connections 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdea010f070 0x7fdea01a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde840a3c40 0x7fde840a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdea00754a0 0x7fdea019f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 --2- 172.21.15.37:0/2180609094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdea0076a70 0x7fdea019f800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:54.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 >> 172.21.15.37:0/2180609094 conn(0x7fdea00fe030 msgr2=0x7fdea0102410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:54.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 shutdown_connections 2024-04-26T01:51:54.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:51:54.081+0000 7fdea5400700 1 -- 172.21.15.37:0/2180609094 wait complete. 2024-04-26T01:51:54.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:54 smithi037 ceph-mon[27906]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:54.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:54.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:54 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:54 smithi037 ceph-mon[27906]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:51:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:54 smithi071 ceph-mon[32578]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:54 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:54 smithi071 ceph-mon[32578]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:51:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:54 smithi045 ceph-mon[32520]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:54 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:54 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:54 smithi045 ceph-mon[32520]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T01:51:54.961 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T01:51:54.967 INFO:tasks.cephadm:Deploying osd.10 on smithi045 with /dev/vg_nvme/lv_1... 2024-04-26T01:51:54.967 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:51:55.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:55 smithi037 ceph-mon[27906]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:51:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:55 smithi037 ceph-mon[27906]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:51:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:55 smithi037 ceph-mon[27906]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:51:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:55 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:55.233 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:51:55.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:55 smithi045 ceph-mon[32520]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:51:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:55 smithi045 ceph-mon[32520]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:51:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:55 smithi045 ceph-mon[32520]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:51:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:55 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:55.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:55.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:51:55.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:55 smithi071 ceph-mon[32578]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T01:51:55.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:55 smithi071 ceph-mon[32578]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T01:51:55.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:55 smithi071 ceph-mon[32578]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-04-26T01:51:55.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:55 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-mon[27906]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:56.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-mon[27906]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:51:56.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-mon[27906]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:51:56.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.189 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.371 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:56 smithi045 ceph-mon[32520]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:56.371 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:56 smithi045 ceph-mon[32520]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:51:56.371 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:56 smithi045 ceph-mon[32520]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:51:56.371 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.371 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:56 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:56 smithi071 ceph-mon[32578]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:56 smithi071 ceph-mon[32578]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-04-26T01:51:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:56 smithi071 ceph-mon[32578]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T01:51:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:56 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:56.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T01:51:56.189+0000 7faa15600700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T01:51:56.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 01:51:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T01:51:56.250+0000 7faa0aa00700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:51:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:57 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:51:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:57 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:51:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:57 smithi045 ceph-mon[32520]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' 2024-04-26T01:51:57.340 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:51:57.340 INFO:teuthology.orchestra.run.smithi045.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:51:57.340 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10+0 records in 2024-04-26T01:51:57.340 INFO:teuthology.orchestra.run.smithi045.stderr:10+0 records out 2024-04-26T01:51:57.340 INFO:teuthology.orchestra.run.smithi045.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00840619 s, 1.2 GB/s 2024-04-26T01:51:57.340 INFO:teuthology.orchestra.run.smithi045.stderr:--> Zapping successful for: 2024-04-26T01:51:57.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:57 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:51:57.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:57 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:51:57.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:57.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:57 smithi071 ceph-mon[32578]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' 2024-04-26T01:51:57.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:57 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:51:57.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:57 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:51:57.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:57.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:57 smithi037 ceph-mon[27906]: from='osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369]' entity='osd.9' 2024-04-26T01:51:57.942 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi045:vg_nvme/lv_1 2024-04-26T01:51:58.197 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T01:51:58.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:58 smithi045 ceph-mon[32520]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:58 smithi045 ceph-mon[32520]: osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] boot 2024-04-26T01:51:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:58 smithi045 ceph-mon[32520]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T01:51:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:58.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:58 smithi071 ceph-mon[32578]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:58 smithi071 ceph-mon[32578]: osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] boot 2024-04-26T01:51:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:58 smithi071 ceph-mon[32578]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T01:51:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:58 smithi037 ceph-mon[27906]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 261 MiB used, 804 GiB / 805 GiB avail 2024-04-26T01:51:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:58 smithi037 ceph-mon[27906]: osd.9 [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] boot 2024-04-26T01:51:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:58 smithi037 ceph-mon[27906]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T01:51:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:51:59.252 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:51:59 smithi045 ceph-mon[32520]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:51:59.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:51:59 smithi071 ceph-mon[32578]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:51:59.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:51:59 smithi037 ceph-mon[27906]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T01:51:59.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 -- 172.21.15.45:0/1125821491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 msgr2=0x7f7534109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:59.738 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 --2- 172.21.15.45:0/1125821491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f7534109df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f752c009fc0 tx=0x7f752c00b330 comp rx=0 tx=0).stop 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 -- 172.21.15.45:0/1125821491 shutdown_connections 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 --2- 172.21.15.45:0/1125821491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75341019a0 0x7f7534101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 --2- 172.21.15.45:0/1125821491 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f7534109df0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 --2- 172.21.15.45:0/1125821491 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7534102ba0 0x7f7534103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 -- 172.21.15.45:0/1125821491 >> 172.21.15.45:0/1125821491 conn(0x7f75340fcf40 msgr2=0x7f75340ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 -- 172.21.15.45:0/1125821491 shutdown_connections 2024-04-26T01:51:59.739 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.740+0000 7f753c2ab700 1 -- 172.21.15.45:0/1125821491 wait complete. 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 Processor -- start 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 -- start start 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75341019a0 0x7f7534197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7534102ba0 0x7f7534198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f753419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f753410d4b0 con 0x7f75341019a0 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f753410d330 con 0x7f7534103540 2024-04-26T01:51:59.741 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.741+0000 7f753c2ab700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f753410d630 con 0x7f7534102ba0 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f753419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f753419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.45:51252/0 (socket says 172.21.15.45:51252) 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 -- 172.21.15.45:0/3250828233 learned_addr learned my addr 172.21.15.45:0/3250828233 (peer_addr_for_me v2:172.21.15.45:0/0) 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f7539846700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7534102ba0 0x7f7534198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 -- 172.21.15.45:0/3250828233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7534102ba0 msgr2=0x7f7534198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7534102ba0 0x7f7534198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 -- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75341019a0 msgr2=0x7f7534197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75341019a0 0x7f7534197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 -- 172.21.15.45:0/3250828233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753406a820 con 0x7f7534103540 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753a848700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f753419f0e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f752c004fb0 tx=0x7f752c009b70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c021070 con 0x7f7534103540 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f752c007c40 con 0x7f7534103540 2024-04-26T01:51:59.742 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753406aab0 con 0x7f7534103540 2024-04-26T01:51:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752c005460 con 0x7f7534103540 2024-04-26T01:51:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.742+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753406aff0 con 0x7f7534103540 2024-04-26T01:51:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.743+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7534069730 con 0x7f7534103540 2024-04-26T01:51:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.745+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f752c024070 con 0x7f7534103540 2024-04-26T01:51:59.743 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.745+0000 7f752b7fe700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f75200a3cc0 0x7f75200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:51:59.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.745+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7f752c0d0e80 con 0x7f7534103540 2024-04-26T01:51:59.744 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.746+0000 7f753a047700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f75200a3cc0 0x7f75200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:51:59.745 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.747+0000 7f753a047700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f75200a3cc0 0x7f75200a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7534102a00 tx=0x7f7530006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:51:59.747 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.749+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f752c09e0f0 con 0x7f7534103540 2024-04-26T01:51:59.964 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:51:59.966+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7534062e80 con 0x7f75200a3cc0 2024-04-26T01:52:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:00 smithi045 ceph-mon[32520]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:52:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:52:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:00.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:00 smithi071 ceph-mon[32578]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:52:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:52:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:00.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:00 smithi037 ceph-mon[27906]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 292 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:52:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:52:00.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:01.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:01 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:01.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:01.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:01 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi045:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:01.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:01.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:01.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:01.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:01 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:02.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:02 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi037 2024-04-26T01:52:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:02 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-26T01:52:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:02 smithi045 ceph-mon[32520]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:02.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:02 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi037 2024-04-26T01:52:02.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:02 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-26T01:52:02.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:02 smithi071 ceph-mon[32578]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:02.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:02 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi037 2024-04-26T01:52:02.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:02 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi037 to 4273M 2024-04-26T01:52:02.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:02 smithi037 ceph-mon[27906]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:03 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/2344386660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]: dispatch 2024-04-26T01:52:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:03 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]: dispatch 2024-04-26T01:52:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:03 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]': finished 2024-04-26T01:52:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:03 smithi045 ceph-mon[32520]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T01:52:03.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:03 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/2344386660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]: dispatch 2024-04-26T01:52:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:03 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]: dispatch 2024-04-26T01:52:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:03 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]': finished 2024-04-26T01:52:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:03 smithi071 ceph-mon[32578]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T01:52:03.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:03.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:03 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/2344386660' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]: dispatch 2024-04-26T01:52:03.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:03 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]: dispatch 2024-04-26T01:52:03.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:03 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2b2d29e-56a1-427e-a87f-911da6c8adf4"}]': finished 2024-04-26T01:52:03.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:03 smithi037 ceph-mon[27906]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T01:52:03.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:04 smithi045 ceph-mon[32520]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:04 smithi045 ceph-mon[32520]: from='client.? 172.21.15.45:0/1855237394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:52:04.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:04 smithi071 ceph-mon[32578]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:04.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:04 smithi071 ceph-mon[32578]: from='client.? 172.21.15.45:0/1855237394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:52:04.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:04 smithi037 ceph-mon[27906]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:04.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:04 smithi037 ceph-mon[27906]: from='client.? 172.21.15.45:0/1855237394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:52:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:06 smithi045 ceph-mon[32520]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:06.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:06 smithi071 ceph-mon[32578]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:06.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:06 smithi037 ceph-mon[27906]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:08 smithi071 ceph-mon[32578]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:08.427 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:08 smithi045 ceph-mon[32520]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:08.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:08 smithi037 ceph-mon[27906]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:10 smithi045 ceph-mon[32520]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:10 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:52:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:10 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:10.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:10 smithi071 ceph-mon[32578]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:10.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:10 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:52:10.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:10 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:10.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:10 smithi037 ceph-mon[27906]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:10.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:10 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T01:52:10.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:10 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:11.616 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:11 smithi045 ceph-mon[32520]: Deploying daemon osd.10 on smithi045 2024-04-26T01:52:11.616 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:11 smithi045 ceph-mon[32520]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:11.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:11 smithi071 ceph-mon[32578]: Deploying daemon osd.10 on smithi045 2024-04-26T01:52:11.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:11 smithi071 ceph-mon[32578]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:11.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:11 smithi037 ceph-mon[27906]: Deploying daemon osd.10 on smithi045 2024-04-26T01:52:11.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:11 smithi037 ceph-mon[27906]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:13.752 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:13 smithi045 ceph-mon[32520]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:13.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:13 smithi071 ceph-mon[32578]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:13.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:13 smithi037 ceph-mon[27906]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:15.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:15 smithi045 ceph-mon[32520]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:15.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:15 smithi071 ceph-mon[32578]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:15.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:15 smithi037 ceph-mon[27906]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:16.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:16 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:16.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:16 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:16.860 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:16 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:16 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:16 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:16.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:16 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:17.003 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:16 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:17.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:16 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:17.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:16 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:17.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:17 smithi045 ceph-mon[32520]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:17.895 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:17 smithi037 ceph-mon[27906]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:17.901 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:17 smithi071 ceph-mon[32578]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:19.102 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.104+0000 7f752b7fe700 1 -- 172.21.15.45:0/3250828233 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7534062e80 con 0x7f75200a3cc0 2024-04-26T01:52:19.102 INFO:teuthology.orchestra.run.smithi045.stdout:Created osd(s) 10 on host 'smithi045' 2024-04-26T01:52:19.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f75200a3cc0 msgr2=0x7f75200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.104 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f75200a3cc0 0x7f75200a6180 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f7534102a00 tx=0x7f7530006d20 comp rx=0 tx=0).stop 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 msgr2=0x7f753419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f753419f0e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f752c004fb0 tx=0x7f752c009b70 comp rx=0 tx=0).stop 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 shutdown_connections 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75341019a0 0x7f7534197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f75200a3cc0 0x7f75200a6180 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7534103540 0x7f753419f0e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 --2- 172.21.15.45:0/3250828233 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7534102ba0 0x7f7534198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 >> 172.21.15.45:0/3250828233 conn(0x7f75340fcf40 msgr2=0x7f7534105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 shutdown_connections 2024-04-26T01:52:19.105 INFO:teuthology.orchestra.run.smithi045.stderr:2024-04-26T01:52:19.106+0000 7f753c2ab700 1 -- 172.21.15.45:0/3250828233 wait complete. 2024-04-26T01:52:19.718 DEBUG:teuthology.orchestra.run.smithi045:osd.10> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T01:52:19.721 INFO:tasks.cephadm:Deploying osd.11 on smithi071 with /dev/vg_nvme/lv_1... 2024-04-26T01:52:19.721 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T01:52:19.967 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: from='osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.372 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:20 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.395 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:20.395 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.395 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.395 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: from='osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:52:20.395 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:52:20.395 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.396 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:20 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:20.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: from='osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:52:20.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T01:52:20.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:20.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:20 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:21.375 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:21 smithi071 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:52:21.375 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:21 smithi071 ceph-mon[32578]: from='osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:52:21.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:21 smithi071 ceph-mon[32578]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:52:21.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:21.376 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:21 smithi071 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:52:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:21 smithi037 ceph-mon[27906]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:52:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:21 smithi037 ceph-mon[27906]: from='osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:52:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:21 smithi037 ceph-mon[27906]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:52:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:21 smithi037 ceph-mon[27906]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:52:21.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-mon[32520]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T01:52:21.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-mon[32520]: from='osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:52:21.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-mon[32520]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T01:52:21.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:21.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-mon[32520]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]: dispatch 2024-04-26T01:52:21.505 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T01:52:21.078+0000 7fd456253700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T01:52:21.505 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 01:52:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T01:52:21.092+0000 7fd452046700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:52:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:22 smithi045 ceph-mon[32520]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:22 smithi045 ceph-mon[32520]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:52:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:22 smithi045 ceph-mon[32520]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:52:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:22 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:22 smithi045 ceph-mon[32520]: osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] boot 2024-04-26T01:52:22.303 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T01:52:22.303 INFO:teuthology.orchestra.run.smithi071.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T01:52:22.303 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10+0 records in 2024-04-26T01:52:22.303 INFO:teuthology.orchestra.run.smithi071.stderr:10+0 records out 2024-04-26T01:52:22.303 INFO:teuthology.orchestra.run.smithi071.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115542 s, 908 MB/s 2024-04-26T01:52:22.304 INFO:teuthology.orchestra.run.smithi071.stderr:--> Zapping successful for: 2024-04-26T01:52:22.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:22 smithi071 ceph-mon[32578]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:22.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:22 smithi071 ceph-mon[32578]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:52:22.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:22 smithi071 ceph-mon[32578]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:52:22.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:22 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:22.360 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:22 smithi071 ceph-mon[32578]: osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] boot 2024-04-26T01:52:22.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:22 smithi037 ceph-mon[27906]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 293 MiB used, 894 GiB / 894 GiB avail 2024-04-26T01:52:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:22 smithi037 ceph-mon[27906]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi045", "root=default"]}]': finished 2024-04-26T01:52:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:22 smithi037 ceph-mon[27906]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T01:52:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:22 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:22 smithi037 ceph-mon[27906]: osd.10 [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] boot 2024-04-26T01:52:22.885 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi071:vg_nvme/lv_1 2024-04-26T01:52:23.129 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:52:23.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:23 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:52:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:23 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:52:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:23 smithi071 ceph-mon[32578]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T01:52:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:23 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:23 smithi071 ceph-mon[32578]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:52:23.436 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:23 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:52:23.436 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:23 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:52:23.436 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:23 smithi045 ceph-mon[32520]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T01:52:23.436 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:23 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:23.437 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:23 smithi045 ceph-mon[32520]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:52:23.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:23 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:52:23.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:23 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:52:23.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:23 smithi037 ceph-mon[27906]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T01:52:23.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:23 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:52:23.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:23 smithi037 ceph-mon[27906]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T01:52:24.232 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.233 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:24.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:24.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:24.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.234 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:24 smithi071 ceph-mon[32578]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:52:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:24 smithi037 ceph-mon[27906]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: pgmap v263: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:24.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:24 smithi045 ceph-mon[32520]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 -- 172.21.15.71:0/3304363959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 msgr2=0x7fbb9c101170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 --2- 172.21.15.71:0/3304363959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c101170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009990 tx=0x7fbb98009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 -- 172.21.15.71:0/3304363959 shutdown_connections 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 --2- 172.21.15.71:0/3304363959 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb9c1046d0 0x7fbb9c10af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 --2- 172.21.15.71:0/3304363959 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c101170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 --2- 172.21.15.71:0/3304363959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9c103d30 0x7fbb9c104190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 -- 172.21.15.71:0/3304363959 >> 172.21.15.71:0/3304363959 conn(0x7fbb9c0fc6d0 msgr2=0x7fbb9c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 -- 172.21.15.71:0/3304363959 shutdown_connections 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 -- 172.21.15.71:0/3304363959 wait complete. 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.698+0000 7fbba2a75700 1 Processor -- start 2024-04-26T01:52:24.699 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 -- start start 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb9c103d30 0x7fbb9c1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9c1046d0 0x7fbb9c1a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10e570 con 0x7fbb9c1046d0 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10e3f0 con 0x7fbb9c103d30 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbb9c10e6f0 con 0x7fbb9c100d50 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:39394/0 (socket says 172.21.15.71:39394) 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 -- 172.21.15.71:0/1718342139 learned_addr learned my addr 172.21.15.71:0/1718342139 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbb93fff700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb9c103d30 0x7fbb9c1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 -- 172.21.15.71:0/1718342139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb9c103d30 msgr2=0x7fbb9c1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb9c103d30 0x7fbb9c1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:24.700 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 -- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9c1046d0 msgr2=0x7fbb9c1a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9c1046d0 0x7fbb9c1a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 -- 172.21.15.71:0/1718342139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb9c1a0810 con 0x7fbb9c100d50 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba0811700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c198bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb980047b0 tx=0x7fbb98009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98027070 con 0x7fbb9c100d50 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.699+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb9c1a0aa0 con 0x7fbb9c100d50 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.700+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb98022470 con 0x7fbb9c100d50 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.700+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9800f6c0 con 0x7fbb9c100d50 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.700+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb9c1a0fd0 con 0x7fbb9c100d50 2024-04-26T01:52:24.701 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.701+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb9c069730 con 0x7fbb9c100d50 2024-04-26T01:52:24.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.702+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbb9802a070 con 0x7fbb9c100d50 2024-04-26T01:52:24.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.702+0000 7fbb91ffb700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fbb840a3c40 0x7fbb840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:24.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.702+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6694+0+0 (secure 0 0 0) 0x7fbb980d1100 con 0x7fbb9c100d50 2024-04-26T01:52:24.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.703+0000 7fbb93fff700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fbb840a3c40 0x7fbb840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:24.702 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.704+0000 7fbb93fff700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fbb840a3c40 0x7fbb840a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c006fd0 tx=0x7fbb8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:24.704 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.706+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb98013070 con 0x7fbb9c100d50 2024-04-26T01:52:24.912 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:24.914+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbb9c062e80 con 0x7fbb840a3c40 2024-04-26T01:52:25.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:25 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi045 2024-04-26T01:52:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:25 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-26T01:52:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:25 smithi071 ceph-mon[32578]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:52:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:52:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:52:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:25 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi045 2024-04-26T01:52:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:25 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-26T01:52:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:25 smithi037 ceph-mon[27906]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:52:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:52:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:52:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:25 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi045 2024-04-26T01:52:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:25 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi045 to 4273M 2024-04-26T01:52:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:25 smithi045 ceph-mon[32520]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T01:52:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T01:52:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T01:52:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:26 smithi071 ceph-mon[32578]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:26.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:26 smithi071 ceph-mon[32578]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:26.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:26 smithi071 ceph-mon[32578]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:52:26.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:26 smithi037 ceph-mon[27906]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:26.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:26 smithi037 ceph-mon[27906]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:26.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:26 smithi037 ceph-mon[27906]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:52:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:26 smithi045 ceph-mon[32520]: pgmap v267: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:26 smithi045 ceph-mon[32520]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi071:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:52:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:26 smithi045 ceph-mon[32520]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T01:52:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:28 smithi071 ceph-mon[32578]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:28 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/884645103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]: dispatch 2024-04-26T01:52:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:28 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]: dispatch 2024-04-26T01:52:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:28 smithi071 ceph-mon[32578]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]': finished 2024-04-26T01:52:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:28 smithi071 ceph-mon[32578]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T01:52:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:28.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:28 smithi037 ceph-mon[27906]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:28 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/884645103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]: dispatch 2024-04-26T01:52:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:28 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]: dispatch 2024-04-26T01:52:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:28 smithi037 ceph-mon[27906]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]': finished 2024-04-26T01:52:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:28 smithi037 ceph-mon[27906]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T01:52:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:28 smithi045 ceph-mon[32520]: pgmap v269: 1 pgs: 1 remapped+peering; 577 KiB data, 320 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:28 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/884645103' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]: dispatch 2024-04-26T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:28 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]: dispatch 2024-04-26T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:28 smithi045 ceph-mon[32520]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16fcf494-d10b-4505-a211-636f44fcf2f1"}]': finished 2024-04-26T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:28 smithi045 ceph-mon[32520]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T01:52:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:29.401 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:29 smithi071 ceph-mon[32578]: from='client.? 172.21.15.71:0/3511908718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:52:29.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:29 smithi037 ceph-mon[27906]: from='client.? 172.21.15.71:0/3511908718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:52:29.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:29 smithi045 ceph-mon[32520]: from='client.? 172.21.15.71:0/3511908718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T01:52:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:30 smithi071 ceph-mon[32578]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-26T01:52:30.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:30 smithi037 ceph-mon[27906]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-26T01:52:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:30 smithi045 ceph-mon[32520]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 131 KiB/s, 0 objects/s recovering 2024-04-26T01:52:32.224 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:32 smithi071 ceph-mon[32578]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:52:32.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:32 smithi037 ceph-mon[27906]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:52:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:32 smithi045 ceph-mon[32520]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T01:52:34.256 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:34 smithi071 ceph-mon[32578]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:52:34.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:34 smithi037 ceph-mon[27906]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:52:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:34 smithi045 ceph-mon[32520]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T01:52:36.315 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:36 smithi071 ceph-mon[32578]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T01:52:36.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:52:36.316 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:36 smithi037 ceph-mon[27906]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T01:52:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:52:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:36 smithi045 ceph-mon[32520]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T01:52:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T01:52:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:37.239 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:37 smithi071 ceph-mon[32578]: Deploying daemon osd.11 on smithi071 2024-04-26T01:52:37.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:37 smithi037 ceph-mon[27906]: Deploying daemon osd.11 on smithi071 2024-04-26T01:52:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:37 smithi045 ceph-mon[32520]: Deploying daemon osd.11 on smithi071 2024-04-26T01:52:38.382 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:38 smithi071 ceph-mon[32578]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:52:38.405 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:38 smithi045 ceph-mon[32520]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:52:38.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:38 smithi037 ceph-mon[27906]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T01:52:40.357 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:40 smithi071 ceph-mon[32578]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:40.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:40 smithi037 ceph-mon[27906]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:40.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:40 smithi045 ceph-mon[32520]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:42.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:41 smithi071 ceph-mon[32578]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:42.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:42.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:42.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:41 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:41 smithi037 ceph-mon[27906]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:41 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:41 smithi045 ceph-mon[32520]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:41 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:44.113 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:43 smithi071 ceph-mon[32578]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:44.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:43 smithi037 ceph-mon[27906]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:43 smithi045 ceph-mon[32520]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:44.799 INFO:teuthology.orchestra.run.smithi071.stdout:Created osd(s) 11 on host 'smithi071' 2024-04-26T01:52:44.799 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.800+0000 7fbb91ffb700 1 -- 172.21.15.71:0/1718342139 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbb9c062e80 con 0x7fbb840a3c40 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fbb840a3c40 msgr2=0x7fbb840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fbb840a3c40 0x7fbb840a6100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbb8c006fd0 tx=0x7fbb8c008040 comp rx=0 tx=0).stop 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 msgr2=0x7fbb9c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c198bb0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb980047b0 tx=0x7fbb98009f90 comp rx=0 tx=0).stop 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 shutdown_connections 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb9c1046d0 0x7fbb9c1a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fbb840a3c40 0x7fbb840a6100 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.801 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbb9c100d50 0x7fbb9c198bb0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 --2- 172.21.15.71:0/1718342139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbb9c103d30 0x7fbb9c1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:44.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 >> 172.21.15.71:0/1718342139 conn(0x7fbb9c0fc6d0 msgr2=0x7fbb9c107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:44.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 shutdown_connections 2024-04-26T01:52:44.802 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:52:44.803+0000 7fbba2a75700 1 -- 172.21.15.71:0/1718342139 wait complete. 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:52:45.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:44 smithi071 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:52:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:44 smithi037 ceph-mon[27906]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:52:45.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:44 smithi045 ceph-mon[32520]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T01:52:45.466 DEBUG:teuthology.orchestra.run.smithi071:osd.11> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T01:52:45.468 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T01:52:45.468 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T01:52:45.928 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:45 smithi071 ceph-mon[32578]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:45 smithi071 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:52:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:45 smithi071 ceph-mon[32578]: from='osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:52:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:45 smithi071 ceph-mon[32578]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:52:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:45 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:45 smithi071 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:52:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:45 smithi045 ceph-mon[32520]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:45 smithi045 ceph-mon[32520]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:52:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:45 smithi045 ceph-mon[32520]: from='osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:52:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:45 smithi045 ceph-mon[32520]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:52:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:45 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:45 smithi045 ceph-mon[32520]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:52:46.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:45 smithi037 ceph-mon[27906]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:46.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:45 smithi037 ceph-mon[27906]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T01:52:46.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:45 smithi037 ceph-mon[27906]: from='osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:52:46.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:45 smithi037 ceph-mon[27906]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T01:52:46.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:45 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:46.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:45 smithi037 ceph-mon[27906]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]: dispatch 2024-04-26T01:52:48.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:47 smithi037 ceph-mon[27906]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:48.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:47 smithi037 ceph-mon[27906]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:52:48.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:47 smithi037 ceph-mon[27906]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:52:48.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:47 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:47 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:47 smithi037 ceph-mon[27906]: from='osd.11 ' entity='osd.11' 2024-04-26T01:52:48.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-mon[32578]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:48.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-mon[32578]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:52:48.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-mon[32578]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:52:48.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.111 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-mon[32578]: from='osd.11 ' entity='osd.11' 2024-04-26T01:52:48.111 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T01:52:47.724+0000 7f5d876a7700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T01:52:48.111 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 01:52:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T01:52:47.856+0000 7f5d7f492700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T01:52:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:47 smithi045 ceph-mon[32520]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:47 smithi045 ceph-mon[32520]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi071", "root=default"]}]': finished 2024-04-26T01:52:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:47 smithi045 ceph-mon[32520]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T01:52:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:47 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:47 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:47 smithi045 ceph-mon[32520]: from='osd.11 ' entity='osd.11' 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 -- 172.21.15.37:0/1478145920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 msgr2=0x7f05c0102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 --2- 172.21.15.37:0/1478145920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c0102ec0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b0a0 tx=0x7f05b000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 -- 172.21.15.37:0/1478145920 shutdown_connections 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 --2- 172.21.15.37:0/1478145920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05c0104640 0x7f05c010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 --2- 172.21.15.37:0/1478145920 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c0102ec0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 --2- 172.21.15.37:0/1478145920 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0103ca0 0x7f05c0104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 -- 172.21.15.37:0/1478145920 >> 172.21.15.37:0/1478145920 conn(0x7f05c00fe040 msgr2=0x7f05c0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.324+0000 7f05c7a00700 1 -- 172.21.15.37:0/1478145920 shutdown_connections 2024-04-26T01:52:48.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.325+0000 7f05c7a00700 1 -- 172.21.15.37:0/1478145920 wait complete. 2024-04-26T01:52:48.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 Processor -- start 2024-04-26T01:52:48.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 -- start start 2024-04-26T01:52:48.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c019ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0103ca0 0x7f05c019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05c0104640 0x7f05c01a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010e530 con 0x7f05c0104640 2024-04-26T01:52:48.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010e3b0 con 0x7f05c0103ca0 2024-04-26T01:52:48.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.326+0000 7f05c7a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f05c010e6b0 con 0x7f05c0102aa0 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c019ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c019ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50762/0 (socket says 172.21.15.37:50762) 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 -- 172.21.15.37:0/2861362855 learned_addr learned my addr 172.21.15.37:0/2861362855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05b7e00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0103ca0 0x7f05c019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05c0104640 0x7f05c01a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 -- 172.21.15.37:0/2861362855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0103ca0 msgr2=0x7f05c019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0103ca0 0x7f05c019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 -- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05c0104640 msgr2=0x7f05c01a23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05c0104640 0x7f05c01a23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.327+0000 7f05c5000700 1 -- 172.21.15.37:0/2861362855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c01a2a80 con 0x7f05c0102aa0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.328+0000 7f05c5000700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c019ae20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b070 tx=0x7f05b0004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.328+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b0021020 con 0x7f05c0102aa0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.328+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05b0007b80 con 0x7f05c0102aa0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.328+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b001b690 con 0x7f05c0102aa0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.328+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c01a2d10 con 0x7f05c0102aa0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.328+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c01a31c0 con 0x7f05c0102aa0 2024-04-26T01:52:48.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.329+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c0069730 con 0x7f05c0102aa0 2024-04-26T01:52:48.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.331+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f05b0024020 con 0x7f05c0102aa0 2024-04-26T01:52:48.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.333+0000 7f05be000700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05a40a3cc0 0x7f05a40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:48.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.333+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 6847+0+0 (secure 0 0 0) 0x7f05b000f020 con 0x7f05c0102aa0 2024-04-26T01:52:48.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.333+0000 7f05b7e00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05a40a3cc0 0x7f05a40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:48.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.334+0000 7f05b7e00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05a40a3cc0 0x7f05a40a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009c60 tx=0x7f05a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:48.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.335+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f05b0007d30 con 0x7f05c0102aa0 2024-04-26T01:52:48.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.587+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f05c01a35f0 con 0x7f05c0102aa0 2024-04-26T01:52:48.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.588+0000 7f05be000700 1 -- 172.21.15.37:0/2861362855 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f05b009e160 con 0x7f05c0102aa0 2024-04-26T01:52:48.587 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.591+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05a40a3cc0 msgr2=0x7f05a40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.591+0000 7f05c7a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05a40a3cc0 0x7f05a40a6180 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009c60 tx=0x7f05a8009260 comp rx=0 tx=0).stop 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.591+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 msgr2=0x7f05c019ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.591+0000 7f05c7a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c019ae20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b070 tx=0x7f05b0004ce0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 shutdown_connections 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05c0104640 0x7f05c01a23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05a40a3cc0 0x7f05a40a6180 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f05c0102aa0 0x7f05c019ae20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 --2- 172.21.15.37:0/2861362855 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f05c0103ca0 0x7f05c019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 >> 172.21.15.37:0/2861362855 conn(0x7f05c00fe040 msgr2=0x7f05c0074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 shutdown_connections 2024-04-26T01:52:48.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:48.592+0000 7f05c7a00700 1 -- 172.21.15.37:0/2861362855 wait complete. 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: purged_snaps scrub starts 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: purged_snaps scrub ok 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:48.921 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:48.922 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:48 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2861362855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:52:49.151 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: purged_snaps scrub starts 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: purged_snaps scrub ok 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.152 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:48 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2861362855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: purged_snaps scrub starts 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: purged_snaps scrub ok 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:52:49.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:52:49.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:52:49.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:48 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2861362855' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:52:49.277 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":80,"num_osds":12,"num_up_osds":11,"osd_up_since":1714096342,"num_in_osds":12,"osd_in_since":1714096347,"num_remapped_pgs":0} 2024-04-26T01:52:49.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:49 smithi037 ceph-mon[27906]: Detected new or changed devices on smithi071 2024-04-26T01:52:49.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:49 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-26T01:52:49.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:49 smithi037 ceph-mon[27906]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:49.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:49 smithi037 ceph-mon[27906]: osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] boot 2024-04-26T01:52:49.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:49 smithi037 ceph-mon[27906]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T01:52:49.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:50.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:49 smithi045 ceph-mon[32520]: Detected new or changed devices on smithi071 2024-04-26T01:52:50.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:49 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-26T01:52:50.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:49 smithi045 ceph-mon[32520]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:50.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:49 smithi045 ceph-mon[32520]: osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] boot 2024-04-26T01:52:50.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:49 smithi045 ceph-mon[32520]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T01:52:50.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:49 smithi071 ceph-mon[32578]: Detected new or changed devices on smithi071 2024-04-26T01:52:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:49 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi071 to 4272M 2024-04-26T01:52:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:49 smithi071 ceph-mon[32578]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 321 MiB used, 983 GiB / 983 GiB avail 2024-04-26T01:52:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:49 smithi071 ceph-mon[32578]: osd.11 [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] boot 2024-04-26T01:52:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:49 smithi071 ceph-mon[32578]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T01:52:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:52:50.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T01:52:50.750 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:51.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:50 smithi071 ceph-mon[32578]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:52:51.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:50 smithi037 ceph-mon[27906]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:52:51.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:50 smithi045 ceph-mon[32520]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T01:52:52.085 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:51 smithi037 ceph-mon[27906]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:52.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:51 smithi071 ceph-mon[32578]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:52.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:51 smithi045 ceph-mon[32520]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:53.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 -- 172.21.15.37:0/1849034091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 msgr2=0x7fca5c10a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:53.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 --2- 172.21.15.37:0/1849034091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c10a740 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fca48009990 tx=0x7fca48009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 -- 172.21.15.37:0/1849034091 shutdown_connections 2024-04-26T01:52:53.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 --2- 172.21.15.37:0/1849034091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 0x7fca5c109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 --2- 172.21.15.37:0/1849034091 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c10ac80 0x7fca5c111530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 --2- 172.21.15.37:0/1849034091 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c10a740 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 -- 172.21.15.37:0/1849034091 >> 172.21.15.37:0/1849034091 conn(0x7fca5c0780b0 msgr2=0x7fca5c07a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 -- 172.21.15.37:0/1849034091 shutdown_connections 2024-04-26T01:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.468+0000 7fca64200700 1 -- 172.21.15.37:0/1849034091 wait complete. 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.469+0000 7fca64200700 1 Processor -- start 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.469+0000 7fca64200700 1 -- start start 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca64200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 0x7fca5c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca64200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c19f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca64200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c10ac80 0x7fca5c1a67b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca64200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c114bb0 con 0x7fca5c1090e0 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca64200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c114a30 con 0x7fca5c10ac80 2024-04-26T01:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca64200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c114d30 con 0x7fca5c10a2e0 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca61800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 0x7fca5c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca60e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c19f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca60e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c19f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:59874/0 (socket says 172.21.15.37:59874) 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca62200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c10ac80 0x7fca5c1a67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca60e00700 1 -- 172.21.15.37:0/1697410027 learned_addr learned my addr 172.21.15.37:0/1697410027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.470+0000 7fca61800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 0x7fca5c19f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37474/0 (socket says 172.21.15.37:37474) 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca60e00700 1 -- 172.21.15.37:0/1697410027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c10ac80 msgr2=0x7fca5c1a67b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca60e00700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c10ac80 0x7fca5c1a67b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca60e00700 1 -- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 msgr2=0x7fca5c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca60e00700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 0x7fca5c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca60e00700 1 -- 172.21.15.37:0/1697410027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c1a6e50 con 0x7fca5c10a2e0 2024-04-26T01:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca60e00700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c19f730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca48000c00 tx=0x7fca480057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48026070 con 0x7fca5c10a2e0 2024-04-26T01:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.471+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca48021410 con 0x7fca5c10a2e0 2024-04-26T01:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.472+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c1a70e0 con 0x7fca5c10a2e0 2024-04-26T01:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.472+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca48017650 con 0x7fca5c10a2e0 2024-04-26T01:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.472+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c1a7590 con 0x7fca5c10a2e0 2024-04-26T01:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.473+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c069730 con 0x7fca5c10a2e0 2024-04-26T01:52:53.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.475+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fca48029070 con 0x7fca5c10a2e0 2024-04-26T01:52:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.476+0000 7fca5a000700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fca440a3cc0 0x7fca440a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.476+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fca480d0fc0 con 0x7fca5c10a2e0 2024-04-26T01:52:53.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.477+0000 7fca61800700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fca440a3cc0 0x7fca440a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:53.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.477+0000 7fca61800700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fca440a3cc0 0x7fca440a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca5c10a140 tx=0x7fca50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:53.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.479+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca48013070 con 0x7fca5c10a2e0 2024-04-26T01:52:53.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.712+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fca5c02df80 con 0x7fca5c10a2e0 2024-04-26T01:52:53.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.713+0000 7fca5a000700 1 -- 172.21.15.37:0/1697410027 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fca4809de30 con 0x7fca5c10a2e0 2024-04-26T01:52:53.712 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:52:53.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.716+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fca440a3cc0 msgr2=0x7fca440a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:53.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.716+0000 7fca64200700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fca440a3cc0 0x7fca440a6180 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca5c10a140 tx=0x7fca50009260 comp rx=0 tx=0).stop 2024-04-26T01:52:53.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.716+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 msgr2=0x7fca5c19f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:53.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.716+0000 7fca64200700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c19f730 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fca48000c00 tx=0x7fca480057c0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 shutdown_connections 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca5c1090e0 0x7fca5c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fca440a3cc0 0x7fca440a6180 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fca5c10ac80 0x7fca5c1a67b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 --2- 172.21.15.37:0/1697410027 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fca5c10a2e0 0x7fca5c19f730 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 >> 172.21.15.37:0/1697410027 conn(0x7fca5c0780b0 msgr2=0x7fca5c10d630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 shutdown_connections 2024-04-26T01:52:53.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:53.717+0000 7fca64200700 1 -- 172.21.15.37:0/1697410027 wait complete. 2024-04-26T01:52:54.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:53 smithi037 ceph-mon[27906]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:54.102 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:53 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1697410027' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:52:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:53 smithi071 ceph-mon[32578]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:53 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1697410027' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:52:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:53 smithi045 ceph-mon[32520]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:53 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1697410027' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T01:52:54.472 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"num_osds":12,"num_up_osds":12,"osd_up_since":1714096368,"num_in_osds":12,"osd_in_since":1714096347,"num_remapped_pgs":0} 2024-04-26T01:52:54.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:52:54.929 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:52:55.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:55 smithi037 ceph-mon[27906]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:55 smithi071 ceph-mon[32578]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:56.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:55 smithi045 ceph-mon[32520]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 -- 172.21.15.37:0/3179423045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 msgr2=0x7f9a4006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 --2- 172.21.15.37:0/3179423045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4006b670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009990 tx=0x7f9a2c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 -- 172.21.15.37:0/3179423045 shutdown_connections 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 --2- 172.21.15.37:0/3179423045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a4006bbb0 0x7f9a4010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 --2- 172.21.15.37:0/3179423045 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a40108a60 0x7f9a40108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 --2- 172.21.15.37:0/3179423045 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4006b670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.268+0000 7f9a45800700 1 -- 172.21.15.37:0/3179423045 >> 172.21.15.37:0/3179423045 conn(0x7f9a400789e0 msgr2=0x7f9a40078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.269+0000 7f9a45800700 1 -- 172.21.15.37:0/3179423045 shutdown_connections 2024-04-26T01:52:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.269+0000 7f9a45800700 1 -- 172.21.15.37:0/3179423045 wait complete. 2024-04-26T01:52:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.270+0000 7f9a45800700 1 Processor -- start 2024-04-26T01:52:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.270+0000 7f9a45800700 1 -- start start 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a45800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4019b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a45800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a4006bbb0 0x7f9a4019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a45800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a40108a60 0x7f9a4019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a45800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112cf0 con 0x7f9a4006bbb0 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a45800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112b70 con 0x7f9a4006b1f0 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a45800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112e70 con 0x7f9a40108a60 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a3ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4019b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a3ea00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4019b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:51412/0 (socket says 172.21.15.37:51412) 2024-04-26T01:52:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a3ea00700 1 -- 172.21.15.37:0/2243524840 learned_addr learned my addr 172.21.15.37:0/2243524840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:52:57.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a3e000700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a4006bbb0 0x7f9a4019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:57.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.271+0000 7f9a3f400700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a40108a60 0x7f9a4019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a3ea00700 1 -- 172.21.15.37:0/2243524840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a40108a60 msgr2=0x7f9a4019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a3ea00700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a40108a60 0x7f9a4019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a3ea00700 1 -- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a4006bbb0 msgr2=0x7f9a4019b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a3ea00700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a4006bbb0 0x7f9a4019b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a3ea00700 1 -- 172.21.15.37:0/2243524840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401a03f0 con 0x7f9a4006b1f0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a3ea00700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4019b3e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e9a0 tx=0x7f9a3000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3000ca10 con 0x7f9a4006b1f0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3000cbb0 con 0x7f9a4006b1f0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a401a06e0 con 0x7f9a4006b1f0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30018680 con 0x7f9a4006b1f0 2024-04-26T01:52:57.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.272+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401a7f60 con 0x7f9a4006b1f0 2024-04-26T01:52:57.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.273+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a40069730 con 0x7f9a4006b1f0 2024-04-26T01:52:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.275+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a30010430 con 0x7f9a4006b1f0 2024-04-26T01:52:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.277+0000 7f9a37400700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a240a3cc0 0x7f9a240a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:52:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.277+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9a300d0090 con 0x7f9a4006b1f0 2024-04-26T01:52:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.277+0000 7f9a3e000700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a240a3cc0 0x7f9a240a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:52:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.278+0000 7f9a3e000700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a240a3cc0 0x7f9a240a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a401080c0 tx=0x7f9a2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:52:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.281+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a3009ce70 con 0x7f9a4006b1f0 2024-04-26T01:52:57.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.514+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a4002df80 con 0x7f9a4006b1f0 2024-04-26T01:52:57.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.516+0000 7f9a37400700 1 -- 172.21.15.37:0/2243524840 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18360 (secure 0 0 0) 0x7f9a3009c770 con 0x7f9a4006b1f0 2024-04-26T01:52:57.515 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:52:57.515 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":82,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:52:49.812629+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a240a3cc0 msgr2=0x7f9a240a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a240a3cc0 0x7f9a240a6180 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9a401080c0 tx=0x7f9a2c005fd0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 msgr2=0x7f9a4019b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4019b3e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9a3000e9a0 tx=0x7f9a3000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 shutdown_connections 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a240a3cc0 0x7f9a240a6180 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a40108a60 0x7f9a4019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a4006b1f0 0x7f9a4019b3e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 --2- 172.21.15.37:0/2243524840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a4006bbb0 0x7f9a4019b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:52:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 >> 172.21.15.37:0/2243524840 conn(0x7f9a400789e0 msgr2=0x7f9a4010de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:52:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.519+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 shutdown_connections 2024-04-26T01:52:57.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:52:57.520+0000 7f9a45800700 1 -- 172.21.15.37:0/2243524840 wait complete. 2024-04-26T01:52:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:57 smithi071 ceph-mon[32578]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:57 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2243524840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:58.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:57 smithi037 ceph-mon[27906]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:58.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:57 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2243524840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:57 smithi045 ceph-mon[32520]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:52:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:57 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2243524840' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:52:58.427 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T01:48:32.516892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T01:52:58.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T01:52:58.911 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:00.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:52:59 smithi037 ceph-mon[27906]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:00.147 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:52:59 smithi071 ceph-mon[32578]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:52:59 smithi045 ceph-mon[32520]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 -- 172.21.15.37:0/3918365645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50102a90 msgr2=0x7fcc50102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 --2- 172.21.15.37:0/3918365645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50102a90 0x7fcc50102f10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fcc38009990 tx=0x7fcc38009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 -- 172.21.15.37:0/3918365645 shutdown_connections 2024-04-26T01:53:01.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 --2- 172.21.15.37:0/3918365645 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc50103450 0x7fcc5010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 --2- 172.21.15.37:0/3918365645 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc50108a50 0x7fcc50108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 --2- 172.21.15.37:0/3918365645 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50102a90 0x7fcc50102f10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 -- 172.21.15.37:0/3918365645 >> 172.21.15.37:0/3918365645 conn(0x7fcc50079ba0 msgr2=0x7fcc50079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:01.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 -- 172.21.15.37:0/3918365645 shutdown_connections 2024-04-26T01:53:01.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.178+0000 7fcc56600700 1 -- 172.21.15.37:0/3918365645 wait complete. 2024-04-26T01:53:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.179+0000 7fcc56600700 1 Processor -- start 2024-04-26T01:53:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.179+0000 7fcc56600700 1 -- start start 2024-04-26T01:53:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc56600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc50102a90 0x7fcc5019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc56600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 0x7fcc5019b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc56600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc50108a50 0x7fcc5019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc56600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50112c50 con 0x7fcc50103450 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc56600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50112ad0 con 0x7fcc50108a50 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc56600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcc50112dd0 con 0x7fcc50102a90 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc47e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 0x7fcc5019b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc47e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 0x7fcc5019b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54596/0 (socket says 172.21.15.37:54596) 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc47e00700 1 -- 172.21.15.37:0/3534227443 learned_addr learned my addr 172.21.15.37:0/3534227443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc4fe00700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc50108a50 0x7fcc5019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.180+0000 7fcc4f400700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc50102a90 0x7fcc5019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc47e00700 1 -- 172.21.15.37:0/3534227443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc50102a90 msgr2=0x7fcc5019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc47e00700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc50102a90 0x7fcc5019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc47e00700 1 -- 172.21.15.37:0/3534227443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc50108a50 msgr2=0x7fcc5019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc47e00700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc50108a50 0x7fcc5019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc47e00700 1 -- 172.21.15.37:0/3534227443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc501a0410 con 0x7fcc50103450 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc47e00700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 0x7fcc5019b940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc38000c00 tx=0x7fcc380057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38027070 con 0x7fcc50103450 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc501a06a0 con 0x7fcc50103450 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc38022470 con 0x7fcc50103450 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.181+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc38017650 con 0x7fcc50103450 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.182+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc501a7f80 con 0x7fcc50103450 2024-04-26T01:53:01.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.183+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc50069730 con 0x7fcc50103450 2024-04-26T01:53:01.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.185+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcc3802a070 con 0x7fcc50103450 2024-04-26T01:53:01.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.187+0000 7fcc4cc00700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fcc340a3c40 0x7fcc340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:01.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.187+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcc380d0c50 con 0x7fcc50103450 2024-04-26T01:53:01.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.187+0000 7fcc4f400700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fcc340a3c40 0x7fcc340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:01.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.188+0000 7fcc4f400700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fcc340a3c40 0x7fcc340a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009be0 tx=0x7fcc40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:01.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.192+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc38013070 con 0x7fcc50103450 2024-04-26T01:53:01.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.421+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcc5019c760 con 0x7fcc50103450 2024-04-26T01:53:01.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.423+0000 7fcc4cc00700 1 -- 172.21.15.37:0/3534227443 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fcc3809dac0 con 0x7fcc50103450 2024-04-26T01:53:01.422 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T01:53:01.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fcc340a3c40 msgr2=0x7fcc340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fcc340a3c40 0x7fcc340a6100 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009be0 tx=0x7fcc40009260 comp rx=0 tx=0).stop 2024-04-26T01:53:01.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 msgr2=0x7fcc5019b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:01.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 0x7fcc5019b940 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcc38000c00 tx=0x7fcc380057c0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 shutdown_connections 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fcc340a3c40 0x7fcc340a6100 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcc50108a50 0x7fcc5019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcc50102a90 0x7fcc5019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 --2- 172.21.15.37:0/3534227443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc50103450 0x7fcc5019b940 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 >> 172.21.15.37:0/3534227443 conn(0x7fcc50079ba0 msgr2=0x7fcc5010df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.426+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 shutdown_connections 2024-04-26T01:53:01.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:01.427+0000 7fcc56600700 1 -- 172.21.15.37:0/3534227443 wait complete. 2024-04-26T01:53:02.132 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:01 smithi037 ceph-mon[27906]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:02.132 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:01 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3534227443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:53:02.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:01 smithi071 ceph-mon[32578]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:02.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:01 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3534227443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:53:02.206 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-04-26T01:53:02.206 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-04-26T01:53:02.206 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.b on smithi045 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.e on smithi045 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.h on smithi045 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.k on smithi045 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.c on smithi071 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.f on smithi071 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.i on smithi071 2024-04-26T01:53:02.207 INFO:tasks.cephadm:Adding mds.l on smithi071 2024-04-26T01:53:02.207 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l' 2024-04-26T01:53:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:01 smithi045 ceph-mon[32520]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:01 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3534227443' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T01:53:02.456 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T01:53:04.101 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:03 smithi071 ceph-mon[32578]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 -- 172.21.15.71:0/3930134573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14103c90 msgr2=0x7fab141040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 --2- 172.21.15.71:0/3930134573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14103c90 0x7fab141040f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 -- 172.21.15.71:0/3930134573 shutdown_connections 2024-04-26T01:53:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 --2- 172.21.15.71:0/3930134573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab14102a90 0x7fab14102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.204 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 --2- 172.21.15.71:0/3930134573 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14104630 0x7fab1410aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 --2- 172.21.15.71:0/3930134573 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14103c90 0x7fab141040f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 -- 172.21.15.71:0/3930134573 >> 172.21.15.71:0/3930134573 conn(0x7fab140fe030 msgr2=0x7fab14100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 -- 172.21.15.71:0/3930134573 shutdown_connections 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 -- 172.21.15.71:0/3930134573 wait complete. 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 Processor -- start 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.099+0000 7fab1a717700 1 -- start start 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab14102a90 0x7fab14198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14103c90 0x7fab141991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 0x7fab141a0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab1410e5c0 con 0x7fab14102a90 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab1410e440 con 0x7fab14103c90 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab1410e740 con 0x7fab14104630 2024-04-26T01:53:04.205 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 0x7fab141a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 0x7fab141a0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.71:55014/0 (socket says 172.21.15.71:55014) 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 -- 172.21.15.71:0/3700178386 learned_addr learned my addr 172.21.15.71:0/3700178386 (peer_addr_for_me v2:172.21.15.71:0/0) 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab137fe700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14103c90 0x7fab141991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 -- 172.21.15.71:0/3700178386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14103c90 msgr2=0x7fab141991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14103c90 0x7fab141991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 -- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab14102a90 msgr2=0x7fab14198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab14102a90 0x7fab14198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 -- 172.21.15.71:0/3700178386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab141a08e0 con 0x7fab14104630 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab137fe700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14103c90 0x7fab141991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab18cb4700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 0x7fab141a0210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab0c007ee0 tx=0x7fab0c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c00dbd0 con 0x7fab14104630 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab0c00dd70 con 0x7fab14104630 2024-04-26T01:53:04.206 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab141a0bd0 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c005990 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.100+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab141a1110 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.101+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab14069730 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.102+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab0c01a020 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.103+0000 7fab117fa700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faafc0a3c40 0x7faafc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.103+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fab0c0cfde0 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.103+0000 7fab13fff700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faafc0a3c40 0x7faafc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.104+0000 7fab13fff700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faafc0a3c40 0x7faafc0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab14199b40 tx=0x7fab04006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:04.207 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.107+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab0c010070 con 0x7fab14104630 2024-04-26T01:53:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:03 smithi037 ceph-mon[27906]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:04.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:03 smithi045 ceph-mon[32520]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:04.313 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.315+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}) -- 0x7fab14063140 con 0x7faafc0a3c40 2024-04-26T01:53:04.330 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.332+0000 7fab117fa700 1 -- 172.21.15.71:0/3700178386 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fab14063140 con 0x7faafc0a3c40 2024-04-26T01:53:04.330 INFO:teuthology.orchestra.run.smithi071.stdout:Scheduled mds.all update... 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faafc0a3c40 msgr2=0x7faafc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faafc0a3c40 0x7faafc0a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fab14199b40 tx=0x7fab04006d20 comp rx=0 tx=0).stop 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 msgr2=0x7fab141a0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 0x7fab141a0210 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab0c007ee0 tx=0x7fab0c00d3b0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 shutdown_connections 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab14102a90 0x7fab14198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7faafc0a3c40 0x7faafc0a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab14104630 0x7fab141a0210 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 --2- 172.21.15.71:0/3700178386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab14103c90 0x7fab141991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:04.333 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.334+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 >> 172.21.15.71:0/3700178386 conn(0x7fab140fe030 msgr2=0x7fab141070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:04.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.335+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 shutdown_connections 2024-04-26T01:53:04.334 INFO:teuthology.orchestra.run.smithi071.stderr:2024-04-26T01:53:04.335+0000 7fab1a717700 1 -- 172.21.15.71:0/3700178386 wait complete. 2024-04-26T01:53:04.934 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T01:53:04.942 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T01:53:04.988 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T01:53:05.035 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T01:53:05.082 DEBUG:teuthology.orchestra.run.smithi045:mds.b> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T01:53:05.084 DEBUG:teuthology.orchestra.run.smithi045:mds.e> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T01:53:05.126 DEBUG:teuthology.orchestra.run.smithi045:mds.h> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T01:53:05.170 DEBUG:teuthology.orchestra.run.smithi045:mds.k> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T01:53:05.212 DEBUG:teuthology.orchestra.run.smithi071:mds.c> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T01:53:05.215 DEBUG:teuthology.orchestra.run.smithi071:mds.f> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T01:53:05.258 DEBUG:teuthology.orchestra.run.smithi071:mds.i> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T01:53:05.301 DEBUG:teuthology.orchestra.run.smithi071:mds.l> sudo journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T01:53:05.345 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T01:53:05.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:05.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:05 smithi045 ceph-mon[32520]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:05.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:05 smithi045 ceph-mon[32520]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-04-26T01:53:05.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:05.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:05.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:05 smithi045 ceph-mon[32520]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:05.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:05 smithi071 ceph-mon[32578]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:05.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:05 smithi071 ceph-mon[32578]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-04-26T01:53:05.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:05.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:05.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:05 smithi071 ceph-mon[32578]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:05.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:05 smithi037 ceph-mon[27906]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:53:05.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:05 smithi037 ceph-mon[27906]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-04-26T01:53:05.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:05.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:05.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:05 smithi037 ceph-mon[27906]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:05.778 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:06 smithi037 ceph-mon[27906]: Deploying daemon mds.l on smithi071 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:06 smithi071 ceph-mon[32578]: Deploying daemon mds.l on smithi071 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:06.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:06 smithi045 ceph-mon[32520]: Deploying daemon mds.l on smithi071 2024-04-26T01:53:07.544 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:07 smithi037 ceph-mon[27906]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:07.551 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:07 smithi071 ceph-mon[32578]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:07 smithi045 ceph-mon[32520]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:07.902 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:07 smithi071 systemd[1]: Starting Ceph mds.l for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:08.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 -- 172.21.15.37:0/2542218051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 msgr2=0x7f134c1136d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:08.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 --2- 172.21.15.37:0/2542218051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c1136d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f133400b0a0 tx=0x7f133400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 -- 172.21.15.37:0/2542218051 shutdown_connections 2024-04-26T01:53:08.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 --2- 172.21.15.37:0/2542218051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 --2- 172.21.15.37:0/2542218051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f134c113c10 0x7f134c116000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 --2- 172.21.15.37:0/2542218051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c1136d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 -- 172.21.15.37:0/2542218051 >> 172.21.15.37:0/2542218051 conn(0x7f134c079ba0 msgr2=0x7f134c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:08.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 -- 172.21.15.37:0/2542218051 shutdown_connections 2024-04-26T01:53:08.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.073+0000 7f1350a00700 1 -- 172.21.15.37:0/2542218051 wait complete. 2024-04-26T01:53:08.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.074+0000 7f1350a00700 1 Processor -- start 2024-04-26T01:53:08.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.074+0000 7f1350a00700 1 -- start start 2024-04-26T01:53:08.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1350a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:08.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1350a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1350a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f134c113c10 0x7f134c1a4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1350a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f134c119270 con 0x7f134c074c70 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1350a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f134c1190f0 con 0x7f134c113c10 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1350a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f134c1193f0 con 0x7f134c075590 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1349e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1349e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54616/0 (socket says 172.21.15.37:54616) 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1349e00700 1 -- 172.21.15.37:0/3060157260 learned_addr learned my addr 172.21.15.37:0/3060157260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f1341400700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.075+0000 7f134a800700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f134c113c10 0x7f134c1a4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1341400700 1 -- 172.21.15.37:0/3060157260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f134c113c10 msgr2=0x7f134c1a4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1341400700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f134c113c10 0x7f134c1a4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1341400700 1 -- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 msgr2=0x7f134c19f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1341400700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c19f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1341400700 1 -- 172.21.15.37:0/3060157260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f134c1a4750 con 0x7f134c075590 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1349e00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1341400700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c19fc80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f133400b070 tx=0x7f1334004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334021070 con 0x7f134c075590 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1334007e30 con 0x7f134c075590 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340046a0 con 0x7f134c075590 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134c1a49e0 con 0x7f134c075590 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.076+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134c1ac2c0 con 0x7f134c075590 2024-04-26T01:53:08.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.077+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134c076360 con 0x7f134c075590 2024-04-26T01:53:08.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.079+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1334007980 con 0x7f134c075590 2024-04-26T01:53:08.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.081+0000 7f1343400700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f13300a3cc0 0x7f13300a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:08.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.081+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f133400f020 con 0x7f134c075590 2024-04-26T01:53:08.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.081+0000 7f1349e00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f13300a3cc0 0x7f13300a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:08.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.082+0000 7f1349e00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f13300a3cc0 0x7f13300a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f133c011550 tx=0x7f133c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:08.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.085+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f133402a020 con 0x7f134c075590 2024-04-26T01:53:08.223 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:07 smithi071 podman[62363]: 2024-04-26 01:53:07.860974896 +0000 UTC m=+0.017541784 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:08.223 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:07 smithi071 podman[62363]: 2024-04-26 01:53:07.979930793 +0000 UTC m=+0.136497723 container create b8e9d2b16da61808a3e4a63193d48bfd31abce8d8e0f10f42cc84a6b747fd768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:53:08.223 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:08 smithi071 podman[62363]: 2024-04-26 01:53:08.199255068 +0000 UTC m=+0.355821966 container init b8e9d2b16da61808a3e4a63193d48bfd31abce8d8e0f10f42cc84a6b747fd768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:53:08.223 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:08 smithi071 podman[62363]: 2024-04-26 01:53:08.202114475 +0000 UTC m=+0.358681376 container start b8e9d2b16da61808a3e4a63193d48bfd31abce8d8e0f10f42cc84a6b747fd768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-26T01:53:08.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.366+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f134c1ac490 con 0x7f134c075590 2024-04-26T01:53:08.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.367+0000 7f1343400700 1 -- 172.21.15.37:0/3060157260 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1387 (secure 0 0 0) 0x7f133409e050 con 0x7f134c075590 2024-04-26T01:53:08.366 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:08.366 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:53:08.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f13300a3cc0 msgr2=0x7f13300a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:08.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f13300a3cc0 0x7f13300a6180 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f133c011550 tx=0x7f133c009260 comp rx=0 tx=0).stop 2024-04-26T01:53:08.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 msgr2=0x7f134c19fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:08.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c19fc80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f133400b070 tx=0x7f1334004d00 comp rx=0 tx=0).stop 2024-04-26T01:53:08.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 shutdown_connections 2024-04-26T01:53:08.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f134c074c70 0x7f134c19f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f13300a3cc0 0x7f13300a6180 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f134c113c10 0x7f134c1a4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 --2- 172.21.15.37:0/3060157260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f134c075590 0x7f134c19fc80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:08.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 >> 172.21.15.37:0/3060157260 conn(0x7f134c079ba0 msgr2=0x7f134c10f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:08.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.371+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 shutdown_connections 2024-04-26T01:53:08.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:08.372+0000 7f1350a00700 1 -- 172.21.15.37:0/3060157260 wait complete. 2024-04-26T01:53:08.372 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-04-26T01:53:08.652 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:08 smithi071 ceph-mds[62382]: -- 172.21.15.71:0/1299221179 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555c0987ba40 con 0x555c0984ec00 2024-04-26T01:53:08.652 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l[62378]: starting mds.l at 2024-04-26T01:53:08.652 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:08 smithi071 bash[62363]: b8e9d2b16da61808a3e4a63193d48bfd31abce8d8e0f10f42cc84a6b747fd768 2024-04-26T01:53:08.652 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 01:53:08 smithi071 systemd[1]: Started Ceph mds.l for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:09.403 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.71:6832/10421136,v1:172.21.15.71:6833/10421136] up:boot 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: fsmap 1 up:standby 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3060157260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:09.404 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:09 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.71:6832/10421136,v1:172.21.15.71:6833/10421136] up:boot 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: fsmap 1 up:standby 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3060157260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:09 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:09.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.71:6832/10421136,v1:172.21.15.71:6833/10421136] up:boot 2024-04-26T01:53:09.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: fsmap 1 up:standby 2024-04-26T01:53:09.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:53:09.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3060157260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:09.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:09.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:09.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:09.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:09 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:10.323 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:10 smithi045 ceph-mon[32520]: Deploying daemon mds.k on smithi045 2024-04-26T01:53:10.323 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:10 smithi045 ceph-mon[32520]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:10.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:10 smithi071 ceph-mon[32578]: Deploying daemon mds.k on smithi045 2024-04-26T01:53:10.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:10 smithi071 ceph-mon[32578]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:10.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:10 smithi037 ceph-mon[27906]: Deploying daemon mds.k on smithi045 2024-04-26T01:53:10.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:10 smithi037 ceph-mon[27906]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:10.927 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:10 smithi045 systemd[1]: Starting Ceph mds.k for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:11.219 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:10 smithi045 podman[62715]: 2024-04-26 01:53:10.832545918 +0000 UTC m=+0.017953822 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:10 smithi045 podman[62715]: 2024-04-26 01:53:10.936859416 +0000 UTC m=+0.122267320 container create a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:11 smithi045 podman[62715]: 2024-04-26 01:53:11.139007802 +0000 UTC m=+0.324415707 container init a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:11 smithi045 podman[62715]: 2024-04-26 01:53:11.141765474 +0000 UTC m=+0.327173384 container start a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True) 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:11 smithi045 ceph-mds[62734]: -- 172.21.15.45:0/4093118417 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e07cb93a40 con 0x55e07cb67000 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k[62730]: starting mds.k at 2024-04-26T01:53:11.221 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:11 smithi045 bash[62715]: a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 2024-04-26T01:53:11.504 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 01:53:11 smithi045 systemd[1]: Started Ceph mds.k for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:11.704 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.45:6832/2132222354,v1:172.21.15.45:6833/2132222354] up:boot 2024-04-26T01:53:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: fsmap 2 up:standby 2024-04-26T01:53:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:53:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:12 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.45:6832/2132222354,v1:172.21.15.45:6833/2132222354] up:boot 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: fsmap 2 up:standby 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:12 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.45:6832/2132222354,v1:172.21.15.45:6833/2132222354] up:boot 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: fsmap 2 up:standby 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:12.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:12.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:12.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:12 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:13.406 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:13 smithi037 ceph-mon[27906]: Deploying daemon mds.j on smithi037 2024-04-26T01:53:13.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:13 smithi045 ceph-mon[32520]: Deploying daemon mds.j on smithi037 2024-04-26T01:53:13.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:13 smithi071 ceph-mon[32578]: Deploying daemon mds.j on smithi037 2024-04-26T01:53:14.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:14 smithi037 ceph-mon[27906]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:14.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:14 smithi071 ceph-mon[32578]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:14.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:14 smithi045 ceph-mon[32520]: pgmap v297: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:14.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 -- 172.21.15.37:0/3932954814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080108a50 msgr2=0x7fc080108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 --2- 172.21.15.37:0/3932954814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080108a50 0x7fc080108e30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc070009990 tx=0x7fc070009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 -- 172.21.15.37:0/3932954814 shutdown_connections 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 --2- 172.21.15.37:0/3932954814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080108a50 0x7fc080108e30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 --2- 172.21.15.37:0/3932954814 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc080103450 0x7fc08010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 --2- 172.21.15.37:0/3932954814 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080102a90 0x7fc080102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 -- 172.21.15.37:0/3932954814 >> 172.21.15.37:0/3932954814 conn(0x7fc080079ba0 msgr2=0x7fc080079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 -- 172.21.15.37:0/3932954814 shutdown_connections 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.755+0000 7fc086a00700 1 -- 172.21.15.37:0/3932954814 wait complete. 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.756+0000 7fc086a00700 1 Processor -- start 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.756+0000 7fc086a00700 1 -- start start 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc086a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc080102a90 0x7fc080074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc086a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 0x7fc080075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc086a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080108a50 0x7fc0800757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc086a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc080112c50 con 0x7fc080103450 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc086a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc080112ad0 con 0x7fc080108a50 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc086a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc080112dd0 con 0x7fc080102a90 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 0x7fc080075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 0x7fc080075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40784/0 (socket says 172.21.15.37:40784) 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 -- 172.21.15.37:0/2682229135 learned_addr learned my addr 172.21.15.37:0/2682229135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07fe00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc080102a90 0x7fc080074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc084a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080108a50 0x7fc0800757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 -- 172.21.15.37:0/2682229135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc080102a90 msgr2=0x7fc080074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc080102a90 0x7fc080074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 -- 172.21.15.37:0/2682229135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080108a50 msgr2=0x7fc0800757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.757+0000 7fc07f400700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080108a50 0x7fc0800757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc07f400700 1 -- 172.21.15.37:0/2682229135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0801a7790 con 0x7fc080103450 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc084a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080108a50 0x7fc0800757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:14.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc07f400700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 0x7fc080075280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00e9a0 tx=0x7fc06c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00ca10 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc06c00cbb0 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c018680 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0801a7a80 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.758+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0801a8010 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.759+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc080069730 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.763+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc06c010430 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.765+0000 7fc07cc00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc0640a3cc0 0x7fc0640a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.765+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc06c0d0420 con 0x7fc080103450 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.765+0000 7fc07fe00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc0640a3cc0 0x7fc0640a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.766+0000 7fc07fe00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc0640a3cc0 0x7fc0640a6180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc07000b2e0 tx=0x7fc070005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:14.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:14.768+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc06c09d200 con 0x7fc080103450 2024-04-26T01:53:15.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.052+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc08002df80 con 0x7fc080103450 2024-04-26T01:53:15.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.053+0000 7fc07cc00700 1 -- 172.21.15.37:0/2682229135 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2212 (secure 0 0 0) 0x7fc06c09cb00 con 0x7fc080103450 2024-04-26T01:53:15.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:15.052 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:53:15.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc0640a3cc0 msgr2=0x7fc0640a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:15.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc0640a3cc0 0x7fc0640a6180 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc07000b2e0 tx=0x7fc070005fd0 comp rx=0 tx=0).stop 2024-04-26T01:53:15.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 msgr2=0x7fc080075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:15.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 0x7fc080075280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00e9a0 tx=0x7fc06c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:53:15.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 shutdown_connections 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc0640a3cc0 0x7fc0640a6180 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc080108a50 0x7fc0800757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc080102a90 0x7fc080074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 --2- 172.21.15.37:0/2682229135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc080103450 0x7fc080075280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 >> 172.21.15.37:0/2682229135 conn(0x7fc080079ba0 msgr2=0x7fc0801018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 shutdown_connections 2024-04-26T01:53:15.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:15.058+0000 7fc086a00700 1 -- 172.21.15.37:0/2682229135 wait complete. 2024-04-26T01:53:15.058 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 3 2024-04-26T01:53:15.601 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:15 smithi037 ceph-mon[27906]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:15.602 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:15 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2682229135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:15.651 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:15 smithi071 ceph-mon[32578]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:15.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:15 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2682229135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:15.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:15 smithi045 ceph-mon[32520]: pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:15.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:15 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2682229135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:16.408 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 podman[68274]: 2024-04-26 01:53:16.208116219 +0000 UTC m=+0.193123129 container create 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T01:53:16.958 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 podman[68274]: 2024-04-26 01:53:16.537216675 +0000 UTC m=+0.522223625 container init 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-26T01:53:16.958 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 podman[68274]: 2024-04-26 01:53:16.549550295 +0000 UTC m=+0.534557183 container start 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:53:16.958 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 ceph-mds[68293]: -- 172.21.15.37:0/3887774299 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4e1a8ba40 con 0x55f4e1a5ec00 2024-04-26T01:53:16.958 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j[68289]: starting mds.j at 2024-04-26T01:53:16.959 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 bash[68274]: 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 2024-04-26T01:53:16.959 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 01:53:16 smithi037 systemd[1]: Started Ceph mds.j for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:17.876 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: Deploying daemon mds.d on smithi037 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.37:6834/3616070379,v1:172.21.15.37:6835/3616070379] up:boot 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: fsmap 3 up:standby 2024-04-26T01:53:17.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:17 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: Deploying daemon mds.d on smithi037 2024-04-26T01:53:18.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.37:6834/3616070379,v1:172.21.15.37:6835/3616070379] up:boot 2024-04-26T01:53:18.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: fsmap 3 up:standby 2024-04-26T01:53:18.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:17 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:53:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: pgmap v299: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:18.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:18.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: Deploying daemon mds.d on smithi037 2024-04-26T01:53:18.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.37:6834/3616070379,v1:172.21.15.37:6835/3616070379] up:boot 2024-04-26T01:53:18.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: fsmap 3 up:standby 2024-04-26T01:53:18.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:17 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:53:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:20.132 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:19 smithi037 ceph-mon[27906]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:19 smithi071 ceph-mon[32578]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:20.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:19 smithi045 ceph-mon[32520]: pgmap v300: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:21.208 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 podman[68681]: 2024-04-26 01:53:21.105786883 +0000 UTC m=+0.208410416 container create 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T01:53:21.830 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 podman[68681]: 2024-04-26 01:53:21.572051342 +0000 UTC m=+0.674674847 container init 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T01:53:21.830 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 podman[68681]: 2024-04-26 01:53:21.581978507 +0000 UTC m=+0.684602023 container start 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T01:53:21.830 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 ceph-mds[68724]: -- 172.21.15.37:0/1125556873 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557746815a40 con 0x5577467e9000 2024-04-26T01:53:21.831 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d[68715]: starting mds.d at 2024-04-26T01:53:21.831 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 bash[68681]: 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a 2024-04-26T01:53:21.831 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 01:53:21 smithi037 systemd[1]: Started Ceph mds.d for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:21.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 -- 172.21.15.37:0/3497411513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c101940 msgr2=0x7fde3c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:21.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 --2- 172.21.15.37:0/3497411513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c101940 0x7fde3c101dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fde2c009990 tx=0x7fde2c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 -- 172.21.15.37:0/3497411513 shutdown_connections 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 --2- 172.21.15.37:0/3497411513 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c102300 0x7fde3c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 --2- 172.21.15.37:0/3497411513 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c107990 0x7fde3c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 --2- 172.21.15.37:0/3497411513 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c101940 0x7fde3c101dc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 -- 172.21.15.37:0/3497411513 >> 172.21.15.37:0/3497411513 conn(0x7fde3c0788f0 msgr2=0x7fde3c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 -- 172.21.15.37:0/3497411513 shutdown_connections 2024-04-26T01:53:21.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:21.999+0000 7fde43a00700 1 -- 172.21.15.37:0/3497411513 wait complete. 2024-04-26T01:53:21.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.000+0000 7fde43a00700 1 Processor -- start 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.000+0000 7fde43a00700 1 -- start start 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde43a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 0x7fde3c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde43a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c102300 0x7fde3c19b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde43a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c107990 0x7fde3c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde43a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c111b40 con 0x7fde3c101940 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde43a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c1119c0 con 0x7fde3c102300 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde43a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde3c111cc0 con 0x7fde3c107990 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 0x7fde3c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 0x7fde3c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57566/0 (socket says 172.21.15.37:57566) 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde33e00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c102300 0x7fde3c19b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 -- 172.21.15.37:0/2177731713 learned_addr learned my addr 172.21.15.37:0/2177731713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:22.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c107990 0x7fde3c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 -- 172.21.15.37:0/2177731713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c107990 msgr2=0x7fde3c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c107990 0x7fde3c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 -- 172.21.15.37:0/2177731713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c102300 msgr2=0x7fde3c19b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.001+0000 7fde41000700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c102300 0x7fde3c19b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde41000700 1 -- 172.21.15.37:0/2177731713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde3c1a03e0 con 0x7fde3c101940 2024-04-26T01:53:22.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde41000700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 0x7fde3c19b3d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e9c0 tx=0x7fde2400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde2400cbd0 con 0x7fde3c101940 2024-04-26T01:53:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde2400ee60 con 0x7fde3c101940 2024-04-26T01:53:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde3c1a06d0 con 0x7fde3c101940 2024-04-26T01:53:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde240055c0 con 0x7fde3c101940 2024-04-26T01:53:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.002+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde3c1a7f50 con 0x7fde3c101940 2024-04-26T01:53:22.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.003+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3c069730 con 0x7fde3c101940 2024-04-26T01:53:22.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.006+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde24004750 con 0x7fde3c101940 2024-04-26T01:53:22.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.007+0000 7fde3a000700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde1c0a3c40 0x7fde1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:22.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.007+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fde240d03e0 con 0x7fde3c101940 2024-04-26T01:53:22.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.007+0000 7fde33e00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde1c0a3c40 0x7fde1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:22.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.008+0000 7fde33e00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde1c0a3c40 0x7fde1c0a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde3c101720 tx=0x7fde2c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:22.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.010+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde24053ff0 con 0x7fde3c101940 2024-04-26T01:53:22.151 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:21 smithi071 ceph-mon[32578]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:22.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:21 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:21 smithi037 ceph-mon[27906]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:21 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:21 smithi045 ceph-mon[32520]: pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:21 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.293+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde3c19c730 con 0x7fde3c101940 2024-04-26T01:53:22.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.295+0000 7fde3a000700 1 -- 172.21.15.37:0/2177731713 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3859 (secure 0 0 0) 0x7fde2409d1c0 con 0x7fde3c101940 2024-04-26T01:53:22.293 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:22.293 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T01:53:22.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde1c0a3c40 msgr2=0x7fde1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde1c0a3c40 0x7fde1c0a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fde3c101720 tx=0x7fde2c005fd0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 msgr2=0x7fde3c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 0x7fde3c19b3d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fde2400e9c0 tx=0x7fde2400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 shutdown_connections 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde3c101940 0x7fde3c19b3d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fde1c0a3c40 0x7fde1c0a6100 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde3c107990 0x7fde3c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 --2- 172.21.15.37:0/2177731713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde3c102300 0x7fde3c19b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:22.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 >> 172.21.15.37:0/2177731713 conn(0x7fde3c0788f0 msgr2=0x7fde3c06a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:22.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 shutdown_connections 2024-04-26T01:53:22.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:22.298+0000 7fde43a00700 1 -- 172.21.15.37:0/2177731713 wait complete. 2024-04-26T01:53:22.298 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.37:6836/200842518,v1:172.21.15.37:6837/200842518] up:boot 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: fsmap 4 up:standby 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: Deploying daemon mds.e on smithi045 2024-04-26T01:53:22.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:22 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2177731713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.37:6836/200842518,v1:172.21.15.37:6837/200842518] up:boot 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: fsmap 4 up:standby 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:23.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:23.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: Deploying daemon mds.e on smithi045 2024-04-26T01:53:23.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:22 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2177731713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.37:6836/200842518,v1:172.21.15.37:6837/200842518] up:boot 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: fsmap 4 up:standby 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:23.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: Deploying daemon mds.e on smithi045 2024-04-26T01:53:23.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:22 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2177731713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:24.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:23 smithi071 ceph-mon[32578]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:24.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:23 smithi037 ceph-mon[27906]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:24.253 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:23 smithi045 ceph-mon[32520]: pgmap v302: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:24.253 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 systemd[1]: Starting Ceph mds.e for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:24.504 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 podman[63024]: 2024-04-26 01:53:24.158780131 +0000 UTC m=+0.018418417 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:24.504 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 podman[63024]: 2024-04-26 01:53:24.258455046 +0000 UTC m=+0.118093342 container create 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:53:24.504 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 podman[63024]: 2024-04-26 01:53:24.452317833 +0000 UTC m=+0.311956089 container init 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T01:53:24.504 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 podman[63024]: 2024-04-26 01:53:24.454937325 +0000 UTC m=+0.314575593 container start 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-26T01:53:24.505 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 ceph-mds[63043]: -- 172.21.15.45:0/2769580623 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564103dc7a40 con 0x564103d9a800 2024-04-26T01:53:24.505 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e[63039]: starting mds.e at 2024-04-26T01:53:25.004 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 bash[63024]: 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 2024-04-26T01:53:25.004 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 01:53:24 smithi045 systemd[1]: Started Ceph mds.e for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:25.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:25.680 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: Deploying daemon mds.f on smithi071 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.45:6834/3667220956,v1:172.21.15.45:6835/3667220956] up:boot 2024-04-26T01:53:25.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: fsmap 5 up:standby 2024-04-26T01:53:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:25 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: Deploying daemon mds.f on smithi071 2024-04-26T01:53:25.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.45:6834/3667220956,v1:172.21.15.45:6835/3667220956] up:boot 2024-04-26T01:53:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: fsmap 5 up:standby 2024-04-26T01:53:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:25 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: pgmap v303: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: Deploying daemon mds.f on smithi071 2024-04-26T01:53:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.45:6834/3667220956,v1:172.21.15.45:6835/3667220956] up:boot 2024-04-26T01:53:26.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: fsmap 5 up:standby 2024-04-26T01:53:26.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:25 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:53:27.152 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:26 smithi071 systemd[1]: Starting Ceph mds.f for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:27.408 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 podman[62674]: 2024-04-26 01:53:27.068104607 +0000 UTC m=+0.017622915 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:27.408 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 podman[62674]: 2024-04-26 01:53:27.166017366 +0000 UTC m=+0.115535647 container create 6d15fa8b32d8d41082f95a21f2ea8f946a5bf83fdaaf373fb235a62d0ac915a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-26T01:53:27.408 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 podman[62674]: 2024-04-26 01:53:27.410660086 +0000 UTC m=+0.360178374 container init 6d15fa8b32d8d41082f95a21f2ea8f946a5bf83fdaaf373fb235a62d0ac915a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-26T01:53:27.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:27 smithi037 ceph-mon[27906]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:27.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:27.821 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:27 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:27.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:27 smithi071 ceph-mon[32578]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:27.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:27.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:27 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:27.902 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 podman[62674]: 2024-04-26 01:53:27.413515468 +0000 UTC m=+0.363033753 container start 6d15fa8b32d8d41082f95a21f2ea8f946a5bf83fdaaf373fb235a62d0ac915a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T01:53:27.902 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 ceph-mds[62693]: -- 172.21.15.71:0/1569761257 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f96e7aba40 con 0x55f96e77e800 2024-04-26T01:53:27.903 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f[62689]: starting mds.f at 2024-04-26T01:53:27.903 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 bash[62674]: 6d15fa8b32d8d41082f95a21f2ea8f946a5bf83fdaaf373fb235a62d0ac915a7 2024-04-26T01:53:27.903 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 01:53:27 smithi071 systemd[1]: Started Ceph mds.f for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:27.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 -- 172.21.15.37:0/2009233763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 msgr2=0x7f316810c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 --2- 172.21.15.37:0/2009233763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f316810c9b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f3154009990 tx=0x7f3154009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 -- 172.21.15.37:0/2009233763 shutdown_connections 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 --2- 172.21.15.37:0/2009233763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f316810cfd0 0x7f316810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 --2- 172.21.15.37:0/2009233763 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3168103ba0 0x7f3168103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 --2- 172.21.15.37:0/2009233763 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f316810c9b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 -- 172.21.15.37:0/2009233763 >> 172.21.15.37:0/2009233763 conn(0x7f31680fd3f0 msgr2=0x7f31680ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 -- 172.21.15.37:0/2009233763 shutdown_connections 2024-04-26T01:53:27.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.996+0000 7f316d400700 1 -- 172.21.15.37:0/2009233763 wait complete. 2024-04-26T01:53:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.998+0000 7f316d400700 1 Processor -- start 2024-04-26T01:53:27.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.998+0000 7f316d400700 1 -- start start 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.998+0000 7f316d400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3168103ba0 0x7f3168101800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.998+0000 7f316d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f3168101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.998+0000 7f316d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f316810cfd0 0x7f3168102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f316d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3168112bb0 con 0x7f316810cfd0 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f316d400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3168112a30 con 0x7f3168103ba0 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f316d400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3168112d30 con 0x7f31681044c0 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f3168101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f316ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f316810cfd0 0x7f3168102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f3168101d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:41198/0 (socket says 172.21.15.37:41198) 2024-04-26T01:53:27.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 -- 172.21.15.37:0/3269917943 learned_addr learned my addr 172.21.15.37:0/3269917943 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:27.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167e00700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3168103ba0 0x7f3168101800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 -- 172.21.15.37:0/3269917943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3168103ba0 msgr2=0x7f3168101800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3168103ba0 0x7f3168101800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 -- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f316810cfd0 msgr2=0x7f3168102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f316810cfd0 0x7f3168102280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:27.999+0000 7f3167400700 1 -- 172.21.15.37:0/3269917943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31681a7620 con 0x7f31681044c0 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.000+0000 7f3167400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f3168101d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3154000c00 tx=0x7f3154009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.000+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154026070 con 0x7f31681044c0 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.000+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3154021410 con 0x7f31681044c0 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.000+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31681a78b0 con 0x7f31681044c0 2024-04-26T01:53:27.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.000+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154017650 con 0x7f31681044c0 2024-04-26T01:53:28.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.000+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31681a7d90 con 0x7f31681044c0 2024-04-26T01:53:28.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.001+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3168069730 con 0x7f31681044c0 2024-04-26T01:53:28.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.003+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3154029070 con 0x7f31681044c0 2024-04-26T01:53:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.005+0000 7f3164c00700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f314c0a3c40 0x7f314c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.005+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f31540d0d80 con 0x7f31681044c0 2024-04-26T01:53:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.005+0000 7f3167e00700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f314c0a3c40 0x7f314c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:28.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.006+0000 7f3167e00700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f314c0a3c40 0x7f314c0a6100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3158005950 tx=0x7f315800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:27 smithi045 ceph-mon[32520]: pgmap v304: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:27 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:28.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.013+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f315405dff0 con 0x7f31681044c0 2024-04-26T01:53:28.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.292+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f316802df80 con 0x7f31681044c0 2024-04-26T01:53:28.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.293+0000 7f3164c00700 1 -- 172.21.15.37:0/3269917943 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5509 (secure 0 0 0) 0x7f3154013070 con 0x7f31681044c0 2024-04-26T01:53:28.291 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:28.292 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.296+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f314c0a3c40 msgr2=0x7f314c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.296+0000 7f316d400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f314c0a3c40 0x7f314c0a6100 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3158005950 tx=0x7f315800b410 comp rx=0 tx=0).stop 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.296+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 msgr2=0x7f3168101d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.296+0000 7f316d400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f3168101d40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3154000c00 tx=0x7f3154009510 comp rx=0 tx=0).stop 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 shutdown_connections 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f316810cfd0 0x7f3168102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f314c0a3c40 0x7f314c0a6100 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3168103ba0 0x7f3168101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 --2- 172.21.15.37:0/3269917943 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31681044c0 0x7f3168101d40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 >> 172.21.15.37:0/3269917943 conn(0x7f31680fd3f0 msgr2=0x7f31680fe1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:28.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 shutdown_connections 2024-04-26T01:53:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:28.297+0000 7f316d400700 1 -- 172.21.15.37:0/3269917943 wait complete. 2024-04-26T01:53:28.296 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 7 2024-04-26T01:53:28.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:28.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:28.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:28.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.71:6834/1784845440,v1:172.21.15.71:6835/1784845440] up:boot 2024-04-26T01:53:28.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: fsmap 6 up:standby 2024-04-26T01:53:28.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:28.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:28.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: Deploying daemon mds.c on smithi071 2024-04-26T01:53:28.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:28 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3269917943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.71:6834/1784845440,v1:172.21.15.71:6835/1784845440] up:boot 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: fsmap 6 up:standby 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: Deploying daemon mds.c on smithi071 2024-04-26T01:53:28.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:28 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3269917943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.71:6834/1784845440,v1:172.21.15.71:6835/1784845440] up:boot 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: fsmap 6 up:standby 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: Deploying daemon mds.c on smithi071 2024-04-26T01:53:29.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:28 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3269917943' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:29.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:29 smithi071 ceph-mon[32578]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:29.902 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:29 smithi071 systemd[1]: Starting Ceph mds.c for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:29.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:29 smithi037 ceph-mon[27906]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:30.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:29 smithi045 ceph-mon[32520]: pgmap v305: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:30.255 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 podman[62995]: 2024-04-26 01:53:29.907182807 +0000 UTC m=+0.017927665 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:30.255 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 podman[62995]: 2024-04-26 01:53:30.004302048 +0000 UTC m=+0.115046891 container create 2f4077c0725f89d10bef708b587dca94f5243857c102596e43c11fa29ebab0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T01:53:30.255 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 podman[62995]: 2024-04-26 01:53:30.231801147 +0000 UTC m=+0.342545985 container init 2f4077c0725f89d10bef708b587dca94f5243857c102596e43c11fa29ebab0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T01:53:30.256 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 podman[62995]: 2024-04-26 01:53:30.234385012 +0000 UTC m=+0.345129856 container start 2f4077c0725f89d10bef708b587dca94f5243857c102596e43c11fa29ebab0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-26T01:53:30.652 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 ceph-mds[63015]: -- 172.21.15.71:0/944372924 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56274b6a7a40 con 0x56274b67a800 2024-04-26T01:53:30.652 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c[63011]: starting mds.c at 2024-04-26T01:53:30.652 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 bash[62995]: 2f4077c0725f89d10bef708b587dca94f5243857c102596e43c11fa29ebab0ea 2024-04-26T01:53:30.652 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 01:53:30 smithi071 systemd[1]: Started Ceph mds.c for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:31.192 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:31.646 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: Deploying daemon mds.h on smithi045 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.71:6836/306008379,v1:172.21.15.71:6837/306008379] up:boot 2024-04-26T01:53:31.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: fsmap 7 up:standby 2024-04-26T01:53:31.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:31 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:53:31.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: Deploying daemon mds.h on smithi045 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.71:6836/306008379,v1:172.21.15.71:6837/306008379] up:boot 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: fsmap 7 up:standby 2024-04-26T01:53:31.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:31 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: Deploying daemon mds.h on smithi045 2024-04-26T01:53:31.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: pgmap v306: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:31.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.71:6836/306008379,v1:172.21.15.71:6837/306008379] up:boot 2024-04-26T01:53:31.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: fsmap 7 up:standby 2024-04-26T01:53:31.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:31 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:53:32.754 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:32 smithi045 systemd[1]: Starting Ceph mds.h for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:33.085 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:32 smithi045 podman[63333]: 2024-04-26 01:53:32.679111298 +0000 UTC m=+0.018391088 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:33.085 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:32 smithi045 podman[63333]: 2024-04-26 01:53:32.776633073 +0000 UTC m=+0.115912848 container create a9f2e6a17d7b867d6236654ce8ee96c1b047cb66fdbae26c13046754d906698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-26T01:53:33.085 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:32 smithi045 podman[63333]: 2024-04-26 01:53:32.987192221 +0000 UTC m=+0.326471997 container init a9f2e6a17d7b867d6236654ce8ee96c1b047cb66fdbae26c13046754d906698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-26T01:53:33.086 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:32 smithi045 podman[63333]: 2024-04-26 01:53:32.989718035 +0000 UTC m=+0.328997810 container start a9f2e6a17d7b867d6236654ce8ee96c1b047cb66fdbae26c13046754d906698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T01:53:33.086 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:33 smithi045 ceph-mds[63352]: -- 172.21.15.45:0/214463432 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea857f1a40 con 0x55ea857c5000 2024-04-26T01:53:33.086 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h[63348]: starting mds.h at 2024-04-26T01:53:33.086 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:33 smithi045 bash[63333]: a9f2e6a17d7b867d6236654ce8ee96c1b047cb66fdbae26c13046754d906698e 2024-04-26T01:53:33.504 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 01:53:33 smithi045 systemd[1]: Started Ceph mds.h for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 -- 172.21.15.37:0/1464571783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c104ec0 msgr2=0x7f9a5c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 --2- 172.21.15.37:0/1464571783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c1052a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9a500099c0 tx=0x7f9a50009cd0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 -- 172.21.15.37:0/1464571783 shutdown_connections 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 --2- 172.21.15.37:0/1464571783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c1052a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 --2- 172.21.15.37:0/1464571783 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 --2- 172.21.15.37:0/1464571783 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c105870 0x7f9a5c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.179+0000 7f9a63c00700 1 -- 172.21.15.37:0/1464571783 >> 172.21.15.37:0/1464571783 conn(0x7f9a5c100750 msgr2=0x7f9a5c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:34.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.180+0000 7f9a63c00700 1 -- 172.21.15.37:0/1464571783 shutdown_connections 2024-04-26T01:53:34.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.180+0000 7f9a63c00700 1 -- 172.21.15.37:0/1464571783 wait complete. 2024-04-26T01:53:34.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 Processor -- start 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 -- start start 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a5c105870 0x7f9a5c19b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c114eb0 con 0x7f9a5c109fa0 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c114d30 con 0x7f9a5c104ec0 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.181+0000 7f9a63c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c115030 con 0x7f9a5c105870 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51668/0 (socket says 172.21.15.37:51668) 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 -- 172.21.15.37:0/3564416444 learned_addr learned my addr 172.21.15.37:0/3564416444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:34.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61200700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:34.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a5be00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a5c105870 0x7f9a5c19b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 -- 172.21.15.37:0/3564416444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a5c105870 msgr2=0x7f9a5c19b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a5c105870 0x7f9a5c19b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 -- 172.21.15.37:0/3564416444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c104ec0 msgr2=0x7f9a5c19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61c00700 1 -- 172.21.15.37:0/3564416444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c1a03e0 con 0x7f9a5c109fa0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.182+0000 7f9a61200700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.183+0000 7f9a61c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c19fca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400ef30 tx=0x7f9a5400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.183+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a540092e0 con 0x7f9a5c109fa0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.183+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c1a0670 con 0x7f9a5c109fa0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.183+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a54004bb0 con 0x7f9a5c109fa0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.183+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54004020 con 0x7f9a5c109fa0 2024-04-26T01:53:34.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.183+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c1a7f50 con 0x7f9a5c109fa0 2024-04-26T01:53:34.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.184+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5c069730 con 0x7f9a5c109fa0 2024-04-26T01:53:34.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.187+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9a5401b030 con 0x7f9a5c109fa0 2024-04-26T01:53:34.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.188+0000 7f9a59600700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a440a3c40 0x7f9a440a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:34.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.188+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f9a540cf9b0 con 0x7f9a5c109fa0 2024-04-26T01:53:34.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.189+0000 7f9a61200700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a440a3c40 0x7f9a440a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:34.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.189+0000 7f9a61200700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a440a3c40 0x7f9a440a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a50003760 tx=0x7f9a50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:34.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.193+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a54022020 con 0x7f9a5c109fa0 2024-04-26T01:53:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:34.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:34.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:34.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: Deploying daemon mds.i on smithi071 2024-04-26T01:53:34.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.45:6836/1744923688,v1:172.21.15.45:6837/1744923688] up:boot 2024-04-26T01:53:34.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: fsmap 8 up:standby 2024-04-26T01:53:34.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:53:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: Deploying daemon mds.i on smithi071 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.45:6836/1744923688,v1:172.21.15.45:6837/1744923688] up:boot 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: fsmap 8 up:standby 2024-04-26T01:53:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:53:34.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.478+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a5c02df80 con 0x7f9a5c109fa0 2024-04-26T01:53:34.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.480+0000 7f9a59600700 1 -- 172.21.15.37:0/3564416444 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7156 (secure 0 0 0) 0x7f9a54010040 con 0x7f9a5c109fa0 2024-04-26T01:53:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:34.478 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T01:53:34.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.482+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a440a3c40 msgr2=0x7f9a440a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a440a3c40 0x7f9a440a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9a50003760 tx=0x7f9a50005fb0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 msgr2=0x7f9a5c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c19fca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400ef30 tx=0x7f9a5400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 shutdown_connections 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9a5c109fa0 0x7f9a5c19fca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9a440a3c40 0x7f9a440a6100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9a5c104ec0 0x7f9a5c19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 --2- 172.21.15.37:0/3564416444 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9a5c105870 0x7f9a5c19b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 >> 172.21.15.37:0/3564416444 conn(0x7f9a5c100750 msgr2=0x7f9a5c102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 shutdown_connections 2024-04-26T01:53:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:34.483+0000 7f9a63c00700 1 -- 172.21.15.37:0/3564416444 wait complete. 2024-04-26T01:53:34.482 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: pgmap v307: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: Deploying daemon mds.i on smithi071 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.45:6836/1744923688,v1:172.21.15.45:6837/1744923688] up:boot 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: fsmap 8 up:standby 2024-04-26T01:53:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:53:35.429 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:35 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3564416444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:35.430 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 systemd[1]: Starting Ceph mds.i for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:35.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:35 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3564416444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:35.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:35 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3564416444' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:35.756 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 podman[63300]: 2024-04-26 01:53:35.63420022 +0000 UTC m=+0.101090954 container create 3520e2016e1b6f625873a7e9aefea87577975f97f2349986ecf7cf37f89a9bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-26T01:53:35.757 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 podman[63300]: 2024-04-26 01:53:35.550771153 +0000 UTC m=+0.017661850 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:36.152 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 podman[63300]: 2024-04-26 01:53:35.870648744 +0000 UTC m=+0.337539432 container init 3520e2016e1b6f625873a7e9aefea87577975f97f2349986ecf7cf37f89a9bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:53:36.152 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 podman[63300]: 2024-04-26 01:53:35.873175637 +0000 UTC m=+0.340066325 container start 3520e2016e1b6f625873a7e9aefea87577975f97f2349986ecf7cf37f89a9bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:53:36.153 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 ceph-mds[63319]: -- 172.21.15.71:0/66519827 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558652f0fa40 con 0x558652ee3000 2024-04-26T01:53:36.153 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: starting mds.i at 2024-04-26T01:53:36.153 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 bash[63300]: 3520e2016e1b6f625873a7e9aefea87577975f97f2349986ecf7cf37f89a9bea 2024-04-26T01:53:36.153 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:53:35 smithi071 systemd[1]: Started Ceph mds.i for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:36.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:37.323 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:37.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:37 smithi037 ceph-mon[27906]: Deploying daemon mds.g on smithi037 2024-04-26T01:53:37.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:37 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] up:boot 2024-04-26T01:53:37.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:37 smithi037 ceph-mon[27906]: fsmap 9 up:standby 2024-04-26T01:53:37.399 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:37 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:53:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:37 smithi071 ceph-mon[32578]: Deploying daemon mds.g on smithi037 2024-04-26T01:53:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:37 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] up:boot 2024-04-26T01:53:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:37 smithi071 ceph-mon[32578]: fsmap 9 up:standby 2024-04-26T01:53:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:37 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:53:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:37 smithi045 ceph-mon[32520]: Deploying daemon mds.g on smithi037 2024-04-26T01:53:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:37 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] up:boot 2024-04-26T01:53:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:37 smithi045 ceph-mon[32520]: fsmap 9 up:standby 2024-04-26T01:53:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:37 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:53:38.369 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:38 smithi037 ceph-mon[27906]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:38 smithi045 ceph-mon[32520]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:38.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:38 smithi071 ceph-mon[32578]: pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:39.316 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:39 smithi037 systemd[1]: Starting Ceph mds.g for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:40.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:40 smithi037 ceph-mon[27906]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:40.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:40 smithi045 ceph-mon[32520]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:40 smithi071 ceph-mon[32578]: pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:40.838 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:40 smithi037 podman[69461]: 2024-04-26 01:53:40.417738597 +0000 UTC m=+0.041662176 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:40.838 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:40 smithi037 podman[69461]: 2024-04-26 01:53:40.536468812 +0000 UTC m=+0.160392448 container create 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-26T01:53:41.208 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:41 smithi037 podman[69461]: 2024-04-26 01:53:41.008730447 +0000 UTC m=+0.632654070 container init 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-26T01:53:41.208 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:41 smithi037 podman[69461]: 2024-04-26 01:53:41.0195584 +0000 UTC m=+0.643481985 container start 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T01:53:41.208 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:41 smithi037 ceph-mds[69499]: -- 172.21.15.37:0/2240995155 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e8752f7a40 con 0x55e8752ca800 2024-04-26T01:53:41.208 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g[69491]: starting mds.g at 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 -- 172.21.15.37:0/3919153082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c108a30 msgr2=0x7f230c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 --2- 172.21.15.37:0/3919153082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c108a30 0x7f230c108e10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2300009990 tx=0x7f2300009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 -- 172.21.15.37:0/3919153082 shutdown_connections 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 --2- 172.21.15.37:0/3919153082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c06b0f0 0x7f230c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 --2- 172.21.15.37:0/3919153082 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c108a30 0x7f230c108e10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 --2- 172.21.15.37:0/3919153082 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c06a750 0x7f230c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 -- 172.21.15.37:0/3919153082 >> 172.21.15.37:0/3919153082 conn(0x7f230c0788f0 msgr2=0x7f230c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 -- 172.21.15.37:0/3919153082 shutdown_connections 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.328+0000 7f2313400700 1 -- 172.21.15.37:0/3919153082 wait complete. 2024-04-26T01:53:41.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 Processor -- start 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 -- start start 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 0x7f230c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c06b0f0 0x7f230c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c108a30 0x7f230c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f230c112c00 con 0x7f230c108a30 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f230c112a80 con 0x7f230c06a750 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2313400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f230c112d80 con 0x7f230c06b0f0 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.330+0000 7f2310a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 0x7f230c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 0x7f230c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:38768/0 (socket says 172.21.15.37:38768) 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 -- 172.21.15.37:0/4067821505 learned_addr learned my addr 172.21.15.37:0/4067821505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:41.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2311400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c108a30 0x7f230c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f230be00700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c06b0f0 0x7f230c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 -- 172.21.15.37:0/4067821505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c06b0f0 msgr2=0x7f230c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c06b0f0 0x7f230c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 -- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c108a30 msgr2=0x7f230c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c108a30 0x7f230c19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 -- 172.21.15.37:0/4067821505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c19e1a0 con 0x7f230c06a750 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.331+0000 7f2310a00700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 0x7f230c199190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2300005910 tx=0x7f230000b670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.332+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300027070 con 0x7f230c06a750 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.332+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2300022470 con 0x7f230c06a750 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.332+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c19e430 con 0x7f230c06a750 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.332+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c1a5d10 con 0x7f230c06a750 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.332+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300017650 con 0x7f230c06a750 2024-04-26T01:53:41.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.333+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c069730 con 0x7f230c06a750 2024-04-26T01:53:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.335+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f230002a070 con 0x7f230c06a750 2024-04-26T01:53:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.336+0000 7f2309600700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f22f40a3cc0 0x7f22f40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.336+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f23000d0bf0 con 0x7f230c06a750 2024-04-26T01:53:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.336+0000 7f230be00700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f22f40a3cc0 0x7f22f40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.337+0000 7f230be00700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f22f40a3cc0 0x7f22f40a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f230c078fd0 tx=0x7f22f8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:41.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.340+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2300013070 con 0x7f230c06a750 2024-04-26T01:53:41.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.652+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f230c02df80 con 0x7f230c06a750 2024-04-26T01:53:41.652 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:41 smithi037 bash[69461]: 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 2024-04-26T01:53:41.652 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 01:53:41 smithi037 systemd[1]: Started Ceph mds.g for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:41.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.653+0000 7f2309600700 1 -- 172.21.15.37:0/4067821505 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8809 (secure 0 0 0) 0x7f230009da60 con 0x7f230c06a750 2024-04-26T01:53:41.653 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:41.653 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f22f40a3cc0 msgr2=0x7f22f40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f22f40a3cc0 0x7f22f40a6180 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f230c078fd0 tx=0x7f22f8009260 comp rx=0 tx=0).stop 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 msgr2=0x7f230c199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 0x7f230c199190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2300005910 tx=0x7f230000b670 comp rx=0 tx=0).stop 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 shutdown_connections 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f230c108a30 0x7f230c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f22f40a3cc0 0x7f22f40a6180 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f230c06a750 0x7f230c199190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 --2- 172.21.15.37:0/4067821505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f230c06b0f0 0x7f230c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.656+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 >> 172.21.15.37:0/4067821505 conn(0x7f230c0788f0 msgr2=0x7f230c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:41.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.657+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 shutdown_connections 2024-04-26T01:53:41.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:41.657+0000 7f2313400700 1 -- 172.21.15.37:0/4067821505 wait complete. 2024-04-26T01:53:41.658 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 11 2024-04-26T01:53:42.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.37:6838/1856429738,v1:172.21.15.37:6839/1856429738] up:boot 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: fsmap 10 up:standby 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:42.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:42 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4067821505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.37:6838/1856429738,v1:172.21.15.37:6839/1856429738] up:boot 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: fsmap 10 up:standby 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:42 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4067821505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.37:6838/1856429738,v1:172.21.15.37:6839/1856429738] up:boot 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: fsmap 10 up:standby 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:42.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:42 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4067821505' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:43.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:43 smithi037 ceph-mon[27906]: Deploying daemon mds.a on smithi037 2024-04-26T01:53:43.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:43 smithi045 ceph-mon[32520]: Deploying daemon mds.a on smithi037 2024-04-26T01:53:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:43 smithi071 ceph-mon[32578]: Deploying daemon mds.a on smithi037 2024-04-26T01:53:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:44 smithi037 ceph-mon[27906]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:44.490 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:44 smithi045 ceph-mon[32520]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:44 smithi071 ceph-mon[32578]: pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:45.186 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:44 smithi037 systemd[1]: Starting Ceph mds.a for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:45.458 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 podman[69848]: 2024-04-26 01:53:45.091074035 +0000 UTC m=+0.034441354 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:45.458 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 podman[69848]: 2024-04-26 01:53:45.254875602 +0000 UTC m=+0.198242978 container create 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T01:53:45.715 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:45.885 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 podman[69848]: 2024-04-26 01:53:45.533593679 +0000 UTC m=+0.476961042 container init 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T01:53:45.885 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 podman[69848]: 2024-04-26 01:53:45.545343892 +0000 UTC m=+0.488711219 container start 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-26T01:53:45.885 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 ceph-mds[69891]: -- 172.21.15.37:0/1609060328 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55999da3fa40 con 0x55999da13000 2024-04-26T01:53:45.885 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a[69887]: starting mds.a at 2024-04-26T01:53:45.885 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 bash[69848]: 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f 2024-04-26T01:53:45.885 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 01:53:45 smithi037 systemd[1]: Started Ceph mds.a for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:46.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:46 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:46 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T01:53:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:46 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:47.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:47 smithi071 ceph-mon[32578]: Deploying daemon mds.b on smithi045 2024-04-26T01:53:47.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:47 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.37:6840/2822754454,v1:172.21.15.37:6841/2822754454] up:boot 2024-04-26T01:53:47.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:47 smithi071 ceph-mon[32578]: fsmap 11 up:standby 2024-04-26T01:53:47.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:47 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:53:47.490 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:47 smithi045 ceph-mon[32520]: Deploying daemon mds.b on smithi045 2024-04-26T01:53:47.490 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:47 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.37:6840/2822754454,v1:172.21.15.37:6841/2822754454] up:boot 2024-04-26T01:53:47.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:47 smithi045 ceph-mon[32520]: fsmap 11 up:standby 2024-04-26T01:53:47.491 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:47 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:53:47.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:47 smithi037 ceph-mon[27906]: Deploying daemon mds.b on smithi045 2024-04-26T01:53:47.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:47 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.37:6840/2822754454,v1:172.21.15.37:6841/2822754454] up:boot 2024-04-26T01:53:47.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:47 smithi037 ceph-mon[27906]: fsmap 11 up:standby 2024-04-26T01:53:47.606 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:47 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:53:48.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 -- 172.21.15.37:0/576232322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c102970 msgr2=0x7f336c102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:48.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 --2- 172.21.15.37:0/576232322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c102970 0x7f336c102df0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3354009990 tx=0x7f3354009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 -- 172.21.15.37:0/576232322 shutdown_connections 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 --2- 172.21.15.37:0/576232322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c103330 0x7f336c10f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 --2- 172.21.15.37:0/576232322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 0x7f336c108d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 --2- 172.21.15.37:0/576232322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c102970 0x7f336c102df0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 -- 172.21.15.37:0/576232322 >> 172.21.15.37:0/576232322 conn(0x7f336c0fe4c0 msgr2=0x7f336c1008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 -- 172.21.15.37:0/576232322 shutdown_connections 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.121+0000 7f336b200700 1 -- 172.21.15.37:0/576232322 wait complete. 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.122+0000 7f336b200700 1 Processor -- start 2024-04-26T01:53:48.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.123+0000 7f336b200700 1 -- start start 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.123+0000 7f336b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 0x7f336c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c103330 0x7f336c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 0x7f336c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f336c112c00 con 0x7f336c102970 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f336c112a80 con 0x7f336c103330 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336b200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f336c112d80 con 0x7f336c108970 2024-04-26T01:53:48.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f3369e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 0x7f336c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f3361400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c103330 0x7f336c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f3369e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 0x7f336c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37340/0 (socket says 172.21.15.37:37340) 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f3369e00700 1 -- 172.21.15.37:0/3368106516 learned_addr learned my addr 172.21.15.37:0/3368106516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336a800700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 0x7f336c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.124+0000 7f336a800700 1 -- 172.21.15.37:0/3368106516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c103330 msgr2=0x7f336c075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336a800700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c103330 0x7f336c075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336a800700 1 -- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 msgr2=0x7f336c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:48.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336a800700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 0x7f336c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336a800700 1 -- 172.21.15.37:0/3368106516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336c1a7590 con 0x7f336c108970 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f3369e00700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 0x7f336c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336a800700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 0x7f336c0757c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f336400efc0 tx=0x7f336400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33640092e0 con 0x7f336c108970 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f336c1a7820 con 0x7f336c108970 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3364004bb0 con 0x7f336c108970 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3364004020 con 0x7f336c108970 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.125+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f336c1a7d60 con 0x7f336c108970 2024-04-26T01:53:48.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.126+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f336c10af00 con 0x7f336c108970 2024-04-26T01:53:48.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.128+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f336401b030 con 0x7f336c108970 2024-04-26T01:53:48.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.130+0000 7f3363400700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f33500a3c40 0x7f33500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:48.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.130+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f33640cfac0 con 0x7f336c108970 2024-04-26T01:53:48.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.130+0000 7f3369e00700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f33500a3c40 0x7f33500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:48.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.131+0000 7f3369e00700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f33500a3c40 0x7f33500a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f335c011550 tx=0x7f335c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:48.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.134+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3364022020 con 0x7f336c108970 2024-04-26T01:53:48.201 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:47 smithi045 systemd[1]: Starting Ceph mds.b for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T01:53:48.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.416+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f336c076220 con 0x7f336c108970 2024-04-26T01:53:48.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:48 smithi037 ceph-mon[27906]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:48.417 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:48.417 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:53:48.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.417+0000 7f3363400700 1 -- 172.21.15.37:0/3368106516 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9635 (secure 0 0 0) 0x7f3364010040 con 0x7f336c108970 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.420+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f33500a3c40 msgr2=0x7f33500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.420+0000 7f336b200700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f33500a3c40 0x7f33500a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f335c011550 tx=0x7f335c009260 comp rx=0 tx=0).stop 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.420+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 msgr2=0x7f336c0757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.420+0000 7f336b200700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 0x7f336c0757c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f336400efc0 tx=0x7f336400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 shutdown_connections 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f336c102970 0x7f336c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f33500a3c40 0x7f33500a6100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f336c108970 0x7f336c0757c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 --2- 172.21.15.37:0/3368106516 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f336c103330 0x7f336c075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 >> 172.21.15.37:0/3368106516 conn(0x7f336c0fe4c0 msgr2=0x7f336c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:48.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 shutdown_connections 2024-04-26T01:53:48.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:48.421+0000 7f336b200700 1 -- 172.21.15.37:0/3368106516 wait complete. 2024-04-26T01:53:48.421 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-04-26T01:53:48.453 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:48 smithi045 ceph-mon[32520]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:48.453 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 podman[63658]: 2024-04-26 01:53:48.203461866 +0000 UTC m=+0.104508767 container create f1ba4643264ab94a5c01321acfaccf04204e9dba088da807cb35f89260e82249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T01:53:48.454 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 podman[63658]: 2024-04-26 01:53:48.117633745 +0000 UTC m=+0.018680660 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T01:53:48.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:48 smithi071 ceph-mon[32578]: pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:48.754 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 podman[63658]: 2024-04-26 01:53:48.456011085 +0000 UTC m=+0.357057979 container init f1ba4643264ab94a5c01321acfaccf04204e9dba088da807cb35f89260e82249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T01:53:48.754 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 podman[63658]: 2024-04-26 01:53:48.459003299 +0000 UTC m=+0.360050192 container start f1ba4643264ab94a5c01321acfaccf04204e9dba088da807cb35f89260e82249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T01:53:48.754 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 ceph-mds[63677]: -- 172.21.15.45:0/1757209397 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x565311ceba40 con 0x565311cbf000 2024-04-26T01:53:48.755 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b[63673]: starting mds.b at 2024-04-26T01:53:48.755 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 bash[63658]: f1ba4643264ab94a5c01321acfaccf04204e9dba088da807cb35f89260e82249 2024-04-26T01:53:48.755 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 01:53:48 smithi045 systemd[1]: Started Ceph mds.b for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3368106516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:49 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3368106516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:49 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3368106516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:49.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:49.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:49 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:50 smithi045 ceph-mon[32520]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:50 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.45:6838/60713297,v1:172.21.15.45:6839/60713297] up:boot 2024-04-26T01:53:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:50 smithi045 ceph-mon[32520]: fsmap 12 up:standby 2024-04-26T01:53:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:50 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:53:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:50 smithi071 ceph-mon[32578]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:50 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.45:6838/60713297,v1:172.21.15.45:6839/60713297] up:boot 2024-04-26T01:53:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:50 smithi071 ceph-mon[32578]: fsmap 12 up:standby 2024-04-26T01:53:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:50 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:53:50.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:50 smithi037 ceph-mon[27906]: pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:50 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.45:6838/60713297,v1:172.21.15.45:6839/60713297] up:boot 2024-04-26T01:53:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:50 smithi037 ceph-mon[27906]: fsmap 12 up:standby 2024-04-26T01:53:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:50 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:53:51.395 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:53:51.871 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:51.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:51 smithi071 ceph-mon[32578]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:51.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:51 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:51 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:51 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:51 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:51 smithi037 ceph-mon[27906]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:51 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:51 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:51 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:51.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:51 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:52.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:51 smithi045 ceph-mon[32520]: pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:52.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:51 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:52.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:51 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:52.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:51 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:52.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:51 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:53.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:53.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:53.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:52 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:53.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:53.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:53.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:52 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:53:53.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:52 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.047 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:53 smithi037 ceph-mon[27906]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:54.048 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.048 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.048 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.048 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:53 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:53 smithi071 ceph-mon[32578]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:53 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:53 smithi045 ceph-mon[32520]: pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:53 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:54.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 -- 172.21.15.37:0/1101788827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 msgr2=0x7f47fc06ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 --2- 172.21.15.37:0/1101788827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc06ab30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009990 tx=0x7f47ec009ca0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 -- 172.21.15.37:0/1101788827 shutdown_connections 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 --2- 172.21.15.37:0/1101788827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc06ab30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 --2- 172.21.15.37:0/1101788827 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47fc10f4f0 0x7f47fc1118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 --2- 172.21.15.37:0/1101788827 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47fc06b070 0x7f47fc10efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 -- 172.21.15.37:0/1101788827 >> 172.21.15.37:0/1101788827 conn(0x7f47fc0788f0 msgr2=0x7f47fc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 -- 172.21.15.37:0/1101788827 shutdown_connections 2024-04-26T01:53:54.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.390+0000 7f4802e00700 1 -- 172.21.15.37:0/1101788827 wait complete. 2024-04-26T01:53:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.392+0000 7f4802e00700 1 Processor -- start 2024-04-26T01:53:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.392+0000 7f4802e00700 1 -- start start 2024-04-26T01:53:54.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.392+0000 7f4802e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f4802e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47fc06b070 0x7f47fc19b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f4802e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47fc10f4f0 0x7f47fc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f4802e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc114e70 con 0x7f47fc06a750 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f4802e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc114cf0 con 0x7f47fc10f4f0 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f4802e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc114ff0 con 0x7f47fc06b070 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f47fbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f47fbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc19b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38808/0 (socket says 172.21.15.37:38808) 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f47fbe00700 1 -- 172.21.15.37:0/1751863111 learned_addr learned my addr 172.21.15.37:0/1751863111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f47fb400700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47fc06b070 0x7f47fc19b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f4800e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47fc10f4f0 0x7f47fc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.393+0000 7f47fbe00700 1 -- 172.21.15.37:0/1751863111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47fc06b070 msgr2=0x7f47fc19b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.394+0000 7f47fbe00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47fc06b070 0x7f47fc19b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.394+0000 7f47fbe00700 1 -- 172.21.15.37:0/1751863111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47fc10f4f0 msgr2=0x7f47fc19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.394+0000 7f47fbe00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47fc10f4f0 0x7f47fc19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.394+0000 7f47fbe00700 1 -- 172.21.15.37:0/1751863111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc1a0390 con 0x7f47fc06a750 2024-04-26T01:53:54.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.394+0000 7f47fbe00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc19b380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47ec005910 tx=0x7f47ec00b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.395+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec027070 con 0x7f47fc06a750 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.395+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47ec022470 con 0x7f47fc06a750 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.395+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec017650 con 0x7f47fc06a750 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.395+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47fc1a0620 con 0x7f47fc06a750 2024-04-26T01:53:54.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.395+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47fc1a7f00 con 0x7f47fc06a750 2024-04-26T01:53:54.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.396+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47fc069730 con 0x7f47fc06a750 2024-04-26T01:53:54.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.398+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f47ec02a070 con 0x7f47fc06a750 2024-04-26T01:53:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.400+0000 7f47f8c00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47e00a3cc0 0x7f47e00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.400+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47ec0d10c0 con 0x7f47fc06a750 2024-04-26T01:53:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.400+0000 7f47fb400700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47e00a3cc0 0x7f47e00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:54.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.401+0000 7f47fb400700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47e00a3cc0 0x7f47e00a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f47fc102ee0 tx=0x7f47e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:54.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.403+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47ec013070 con 0x7f47fc06a750 2024-04-26T01:53:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.690+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47fc19c6e0 con 0x7f47fc06a750 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.692+0000 7f47f8c00700 1 -- 172.21.15.37:0/1751863111 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10455 (secure 0 0 0) 0x7f47ec09df30 con 0x7f47fc06a750 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:54.691 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24509,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24514,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T01:53:54.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47e00a3cc0 msgr2=0x7f47e00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47e00a3cc0 0x7f47e00a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f47fc102ee0 tx=0x7f47e8009260 comp rx=0 tx=0).stop 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 msgr2=0x7f47fc19b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc19b380 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f47ec005910 tx=0x7f47ec00b670 comp rx=0 tx=0).stop 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 shutdown_connections 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47fc06a750 0x7f47fc19b380 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47e00a3cc0 0x7f47e00a6180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47fc10f4f0 0x7f47fc19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 --2- 172.21.15.37:0/1751863111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47fc06b070 0x7f47fc19b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 >> 172.21.15.37:0/1751863111 conn(0x7f47fc0788f0 msgr2=0x7f47fc1109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 shutdown_connections 2024-04-26T01:53:54.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:54.695+0000 7f4802e00700 1 -- 172.21.15.37:0/1751863111 wait complete. 2024-04-26T01:53:54.696 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-04-26T01:53:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:54 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1751863111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:54 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1751863111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:54 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1751863111' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:53:55.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-26T01:53:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:55 smithi071 ceph-mon[32578]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:56.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:55 smithi037 ceph-mon[27906]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:55 smithi045 ceph-mon[32520]: pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:53:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:57 smithi071 ceph-mon[32578]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:57 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:57 smithi037 ceph-mon[27906]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:58.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:58.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:58.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:57 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:57 smithi045 ceph-mon[32520]: pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:53:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:53:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:53:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:57 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:53:58.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 -- 172.21.15.37:0/3198663550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f378810e3b0 msgr2=0x7f3788115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 --2- 172.21.15.37:0/3198663550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f378810e3b0 0x7f3788115f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f37800099c0 tx=0x7f3780009cd0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 -- 172.21.15.37:0/3198663550 shutdown_connections 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 --2- 172.21.15.37:0/3198663550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f378810e3b0 0x7f3788115f80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 --2- 172.21.15.37:0/3198663550 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f37881092d0 0x7f37881096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 --2- 172.21.15.37:0/3198663550 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3788109c80 0x7f378810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 -- 172.21.15.37:0/3198663550 >> 172.21.15.37:0/3198663550 conn(0x7f3788076210 msgr2=0x7f3788107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 -- 172.21.15.37:0/3198663550 shutdown_connections 2024-04-26T01:53:58.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.764+0000 7f378d400700 1 -- 172.21.15.37:0/3198663550 wait complete. 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.765+0000 7f378d400700 1 Processor -- start 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.765+0000 7f378d400700 1 -- start start 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f378d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 0x7f378819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f378d400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 0x7f378819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f378d400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f378810e3b0 0x7f37881a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f378d400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f37881192a0 con 0x7f37881092d0 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f378d400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3788119120 con 0x7f3788109c80 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f378d400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3788119420 con 0x7f378810e3b0 2024-04-26T01:53:58.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f377de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 0x7f378819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f3786800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 0x7f378819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f377de00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 0x7f378819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:44388/0 (socket says 172.21.15.37:44388) 2024-04-26T01:53:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f377de00700 1 -- 172.21.15.37:0/3472115219 learned_addr learned my addr 172.21.15.37:0/3472115219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:53:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f3786800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 0x7f378819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38834/0 (socket says 172.21.15.37:38834) 2024-04-26T01:53:58.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.766+0000 7f3787200700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f378810e3b0 0x7f37881a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377de00700 1 -- 172.21.15.37:0/3472115219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f378810e3b0 msgr2=0x7f37881a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377de00700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f378810e3b0 0x7f37881a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377de00700 1 -- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 msgr2=0x7f378819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377de00700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 0x7f378819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377de00700 1 -- 172.21.15.37:0/3472115219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37881a4730 con 0x7f3788109c80 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f3786800700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 0x7f378819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377de00700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 0x7f378819fcb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f377000e970 tx=0x7f377000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770004a30 con 0x7f3788109c80 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f377000cc90 con 0x7f3788109c80 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881a4a20 con 0x7f3788109c80 2024-04-26T01:53:58.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3770005610 con 0x7f3788109c80 2024-04-26T01:53:58.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.767+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881ac2a0 con 0x7f3788109c80 2024-04-26T01:53:58.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.768+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3788069730 con 0x7f3788109c80 2024-04-26T01:53:58.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.771+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f377000c7e0 con 0x7f3788109c80 2024-04-26T01:53:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.772+0000 7f377fe00700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f376c0a3c40 0x7f376c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:53:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.772+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f37700d00c0 con 0x7f3788109c80 2024-04-26T01:53:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.772+0000 7f3786800700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f376c0a3c40 0x7f376c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:53:58.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.773+0000 7f3786800700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f376c0a3c40 0x7f376c0a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fd0 tx=0x7f37780091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:53:58.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:58.777+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f377009cea0 con 0x7f3788109c80 2024-04-26T01:53:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.011+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f378802df80 con 0x7f3788109c80 2024-04-26T01:53:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.011+0000 7f377fe00700 1 -- 172.21.15.37:0/3472115219 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f377009c7a0 con 0x7f3788109c80 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:53:59.011 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-04-26T01:53:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f376c0a3c40 msgr2=0x7f376c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f376c0a3c40 0x7f376c0a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3778009fd0 tx=0x7f37780091f0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 msgr2=0x7f378819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:53:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 0x7f378819fcb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f377000e970 tx=0x7f377000ec80 comp rx=0 tx=0).stop 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 shutdown_connections 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f37881092d0 0x7f378819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f376c0a3c40 0x7f376c0a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f378810e3b0 0x7f37881a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 --2- 172.21.15.37:0/3472115219 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3788109c80 0x7f378819fcb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 >> 172.21.15.37:0/3472115219 conn(0x7f3788076210 msgr2=0x7f37880767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.015+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 shutdown_connections 2024-04-26T01:53:59.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:53:59.016+0000 7f378d400700 1 -- 172.21.15.37:0/3472115219 wait complete. 2024-04-26T01:53:59.932 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T01:53:59.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T01:54:00.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:59 smithi071 ceph-mon[32578]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:00.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:53:59 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3472115219' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:54:00.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:59 smithi037 ceph-mon[27906]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:00.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:53:59 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3472115219' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:54:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:59 smithi045 ceph-mon[32520]: pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:53:59 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3472115219' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T01:54:00.415 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:02.116 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:01 smithi037 ceph-mon[27906]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:01 smithi045 ceph-mon[32520]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:01 smithi071 ceph-mon[32578]: pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 -- 172.21.15.37:0/906548921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec109e90 msgr2=0x7f8fec111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 --2- 172.21.15.37:0/906548921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec109e90 0x7f8fec111a60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009990 tx=0x7f8fe4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 -- 172.21.15.37:0/906548921 shutdown_connections 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 --2- 172.21.15.37:0/906548921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec109e90 0x7f8fec111a60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 --2- 172.21.15.37:0/906548921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 --2- 172.21.15.37:0/906548921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec105760 0x7f8fec1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 -- 172.21.15.37:0/906548921 >> 172.21.15.37:0/906548921 conn(0x7f8fec100620 msgr2=0x7f8fec102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 -- 172.21.15.37:0/906548921 shutdown_connections 2024-04-26T01:54:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.623+0000 7f8feb200700 1 -- 172.21.15.37:0/906548921 wait complete. 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.624+0000 7f8feb200700 1 Processor -- start 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 -- start start 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec105760 0x7f8fec19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 0x7f8fec19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec114d70 con 0x7f8fec109e90 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec114bf0 con 0x7f8fec105760 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8feb200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8fec114ef0 con 0x7f8fec104db0 2024-04-26T01:54:02.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8fe9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.625+0000 7f8fea800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 0x7f8fec19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec19b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50782/0 (socket says 172.21.15.37:50782) 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fea800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 0x7f8fec19fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42248/0 (socket says 172.21.15.37:42248) 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 -- 172.21.15.37:0/2204508112 learned_addr learned my addr 172.21.15.37:0/2204508112 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9400700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec105760 0x7f8fec19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 -- 172.21.15.37:0/2204508112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec105760 msgr2=0x7f8fec19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec105760 0x7f8fec19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 -- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 msgr2=0x7f8fec19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 0x7f8fec19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 -- 172.21.15.37:0/2204508112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fec1a0200 con 0x7f8fec104db0 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fea800700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 0x7f8fec19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.626+0000 7f8fe9e00700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec19b240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00ef10 tx=0x7f8fdc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:02.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.627+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc00c9c0 con 0x7f8fec104db0 2024-04-26T01:54:02.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.627+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc019070 con 0x7f8fec104db0 2024-04-26T01:54:02.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.627+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fec1a0490 con 0x7f8fec104db0 2024-04-26T01:54:02.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.627+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc014480 con 0x7f8fec104db0 2024-04-26T01:54:02.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.627+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fec1a7d70 con 0x7f8fec104db0 2024-04-26T01:54:02.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.628+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fec069730 con 0x7f8fec104db0 2024-04-26T01:54:02.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.630+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fdc012050 con 0x7f8fec104db0 2024-04-26T01:54:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.631+0000 7f8fe2a00700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8fd00a3c40 0x7f8fd00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.632+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8fdc0cff10 con 0x7f8fec104db0 2024-04-26T01:54:02.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.632+0000 7f8fe9400700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8fd00a3c40 0x7f8fd00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:02.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.632+0000 7f8fe9400700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8fd00a3c40 0x7f8fd00a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009680 tx=0x7f8fd8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:02.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.636+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fdc010040 con 0x7f8fec104db0 2024-04-26T01:54:02.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.873+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f8fec02df80 con 0x7f8fec104db0 2024-04-26T01:54:02.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.923+0000 7f8fe2a00700 1 -- 172.21.15.37:0/2204508112 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f8fdc09ccf0 con 0x7f8fec104db0 2024-04-26T01:54:02.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.932+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8fd00a3c40 msgr2=0x7f8fd00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.932+0000 7f8feb200700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8fd00a3c40 0x7f8fd00a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009680 tx=0x7f8fd8009260 comp rx=0 tx=0).stop 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.932+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 msgr2=0x7f8fec19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.932+0000 7f8feb200700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec19b240 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc00ef10 tx=0x7f8fdc00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.933+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 shutdown_connections 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.933+0000 7f8feb200700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fec109e90 0x7f8fec19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.933+0000 7f8feb200700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8fd00a3c40 0x7f8fd00a6100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.933+0000 7f8feb200700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8fec104db0 0x7f8fec19b240 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.933+0000 7f8feb200700 1 --2- 172.21.15.37:0/2204508112 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8fec105760 0x7f8fec19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:02.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.933+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 >> 172.21.15.37:0/2204508112 conn(0x7f8fec100620 msgr2=0x7f8fec102920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:02.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.934+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 shutdown_connections 2024-04-26T01:54:02.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:02.934+0000 7f8feb200700 1 -- 172.21.15.37:0/2204508112 wait complete. 2024-04-26T01:54:02.935 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-04-26T01:54:03.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:02 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2204508112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:54:03.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:02 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:54:03.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:02 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2204508112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:54:03.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:02 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:54:03.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:02 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2204508112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:54:03.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:02 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T01:54:03.830 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T01:54:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:03 smithi037 ceph-mon[27906]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:03 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:54:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:03 smithi037 ceph-mon[27906]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:54:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:03 smithi045 ceph-mon[32520]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:03 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:54:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:03 smithi045 ceph-mon[32520]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:54:04.308 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:03 smithi071 ceph-mon[32578]: pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:03 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T01:54:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:03 smithi071 ceph-mon[32578]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T01:54:05.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:04 smithi037 ceph-mon[27906]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:54:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:04 smithi045 ceph-mon[32520]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:54:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:04 smithi071 ceph-mon[32578]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T01:54:06.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:05 smithi037 ceph-mon[27906]: pgmap v325: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:06.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:05 smithi037 ceph-mon[27906]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:54:06.163 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:05 smithi037 ceph-mon[27906]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:54:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:05 smithi045 ceph-mon[32520]: pgmap v325: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:05 smithi045 ceph-mon[32520]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:54:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:05 smithi045 ceph-mon[32520]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:54:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:05 smithi071 ceph-mon[32578]: pgmap v325: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:05 smithi071 ceph-mon[32578]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:54:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:05 smithi071 ceph-mon[32578]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T01:54:06.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 -- 172.21.15.37:0/842859993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008113740 msgr2=0x7f6008115b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 --2- 172.21.15.37:0/842859993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008113740 0x7f6008115b30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6000009f90 tx=0x7f600000b330 comp rx=0 tx=0).stop 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 -- 172.21.15.37:0/842859993 shutdown_connections 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 --2- 172.21.15.37:0/842859993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6008076a20 0x7f6008074c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 --2- 172.21.15.37:0/842859993 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008113740 0x7f6008115b30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 --2- 172.21.15.37:0/842859993 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f60080751b0 0x7f6008075630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 -- 172.21.15.37:0/842859993 >> 172.21.15.37:0/842859993 conn(0x7f6008079c00 msgr2=0x7f600807a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 -- 172.21.15.37:0/842859993 shutdown_connections 2024-04-26T01:54:06.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.710+0000 7f600d200700 1 -- 172.21.15.37:0/842859993 wait complete. 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.711+0000 7f600d200700 1 Processor -- start 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.711+0000 7f600d200700 1 -- start start 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f600d200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60080751b0 0x7f600819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f600d200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008076a20 0x7f600819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f600d200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008113740 0x7f60081a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f600d200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6008119240 con 0x7f60080751b0 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f600d200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f60081190c0 con 0x7f6008113740 2024-04-26T01:54:06.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f600d200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f60081193c0 con 0x7f6008076a20 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f6005e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60080751b0 0x7f600819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f6005e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60080751b0 0x7f600819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42274/0 (socket says 172.21.15.37:42274) 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f6005e00700 1 -- 172.21.15.37:0/91131734 learned_addr learned my addr 172.21.15.37:0/91131734 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f6006800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008113740 0x7f60081a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.712+0000 7f6005400700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008076a20 0x7f600819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6006800700 1 -- 172.21.15.37:0/91131734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008076a20 msgr2=0x7f600819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6006800700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008076a20 0x7f600819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6006800700 1 -- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60080751b0 msgr2=0x7f600819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6006800700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60080751b0 0x7f600819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6006800700 1 -- 172.21.15.37:0/91131734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60081a4720 con 0x7f6008113740 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6005400700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008076a20 0x7f600819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:06.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f6006800700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008113740 0x7f60081a3fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f600000b6c0 tx=0x7f6000009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000021070 con 0x7f6008113740 2024-04-26T01:54:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.713+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f600000ce80 con 0x7f6008113740 2024-04-26T01:54:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.714+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6000005040 con 0x7f6008113740 2024-04-26T01:54:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.714+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60081a49b0 con 0x7f6008113740 2024-04-26T01:54:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.714+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60081ac290 con 0x7f6008113740 2024-04-26T01:54:06.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.715+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60080762d0 con 0x7f6008113740 2024-04-26T01:54:06.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.717+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6000024070 con 0x7f6008113740 2024-04-26T01:54:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.718+0000 7f5ffea00700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5fec0a3c40 0x7f5fec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:06.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.718+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f600000f020 con 0x7f6008113740 2024-04-26T01:54:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.719+0000 7f6005e00700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5fec0a3c40 0x7f5fec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:06.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.719+0000 7f6005e00700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5fec0a3c40 0x7f5fec0a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009750 tx=0x7f5ff8009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:06.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.723+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f600002b020 con 0x7f6008113740 2024-04-26T01:54:06.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:06.955+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f60081ac460 con 0x7f6008113740 2024-04-26T01:54:07.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.966+0000 7f5ffea00700 1 -- 172.21.15.37:0/91131734 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f600009e1d0 con 0x7f6008113740 2024-04-26T01:54:07.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5fec0a3c40 msgr2=0x7f5fec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:07.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5fec0a3c40 0x7f5fec0a6100 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009750 tx=0x7f5ff8009310 comp rx=0 tx=0).stop 2024-04-26T01:54:07.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008113740 msgr2=0x7f60081a3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:07.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008113740 0x7f60081a3fe0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f600000b6c0 tx=0x7f6000009510 comp rx=0 tx=0).stop 2024-04-26T01:54:07.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 shutdown_connections 2024-04-26T01:54:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60080751b0 0x7f600819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5fec0a3c40 0x7f5fec0a6100 secure :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8009750 tx=0x7f5ff8009310 comp rx=0 tx=0).stop 2024-04-26T01:54:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6008113740 0x7f60081a3fe0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 --2- 172.21.15.37:0/91131734 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6008076a20 0x7f600819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:07.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.973+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 >> 172.21.15.37:0/91131734 conn(0x7f6008079c00 msgr2=0x7f6008112440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.975+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 shutdown_connections 2024-04-26T01:54:07.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:07.975+0000 7f600d200700 1 -- 172.21.15.37:0/91131734 wait complete. 2024-04-26T01:54:07.978 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-04-26T01:54:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:07 smithi037 ceph-mon[27906]: pgmap v327: 33 pgs: 6 creating+peering, 14 active+clean, 13 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:07 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/91131734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:54:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:07 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:54:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:07 smithi045 ceph-mon[32520]: pgmap v327: 33 pgs: 6 creating+peering, 14 active+clean, 13 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:07 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/91131734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:54:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:07 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:54:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:07 smithi071 ceph-mon[32578]: pgmap v327: 33 pgs: 6 creating+peering, 14 active+clean, 13 unknown; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:07 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/91131734' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:54:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:07 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T01:54:08.883 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T01:54:09.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:08 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:54:09.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:08 smithi037 ceph-mon[27906]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:54:09.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:08 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:54:09.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:08 smithi045 ceph-mon[32520]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:54:09.368 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:09.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:08 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T01:54:09.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:08 smithi071 ceph-mon[32578]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T01:54:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:09 smithi045 ceph-mon[32520]: pgmap v329: 97 pgs: 6 creating+peering, 58 unknown, 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:09 smithi045 ceph-mon[32520]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:54:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:09 smithi071 ceph-mon[32578]: pgmap v329: 97 pgs: 6 creating+peering, 58 unknown, 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:09 smithi071 ceph-mon[32578]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:54:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:09 smithi037 ceph-mon[27906]: pgmap v329: 97 pgs: 6 creating+peering, 58 unknown, 33 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:09 smithi037 ceph-mon[27906]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T01:54:11.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:11 smithi037 ceph-mon[27906]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:54:11.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:11 smithi045 ceph-mon[32520]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:54:11.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:11 smithi071 ceph-mon[32578]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T01:54:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 -- 172.21.15.37:0/2939776138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c075590 msgr2=0x7fc90c10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 --2- 172.21.15.37:0/2939776138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c075590 0x7fc90c10ef00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4009990 tx=0x7fc8f4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 -- 172.21.15.37:0/2939776138 shutdown_connections 2024-04-26T01:54:11.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 --2- 172.21.15.37:0/2939776138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 0x7fc90c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 --2- 172.21.15.37:0/2939776138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c074c70 0x7fc90c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 --2- 172.21.15.37:0/2939776138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c075590 0x7fc90c10ef00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 -- 172.21.15.37:0/2939776138 >> 172.21.15.37:0/2939776138 conn(0x7fc90c079ba0 msgr2=0x7fc90c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 -- 172.21.15.37:0/2939776138 shutdown_connections 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.859+0000 7fc911200700 1 -- 172.21.15.37:0/2939776138 wait complete. 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.860+0000 7fc911200700 1 Processor -- start 2024-04-26T01:54:11.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 -- start start 2024-04-26T01:54:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c074c70 0x7fc90c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c075590 0x7fc90c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 0x7fc90c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c114e00 con 0x7fc90c074c70 2024-04-26T01:54:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c114c80 con 0x7fc90c10f440 2024-04-26T01:54:11.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc911200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc90c114f80 con 0x7fc90c075590 2024-04-26T01:54:11.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.861+0000 7fc909e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c074c70 0x7fc90c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc901400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c075590 0x7fc90c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 0x7fc90c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc909e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c074c70 0x7fc90c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36276/0 (socket says 172.21.15.37:36276) 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc909e00700 1 -- 172.21.15.37:0/3437435493 learned_addr learned my addr 172.21.15.37:0/3437435493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc901400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c075590 0x7fc90c19b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34462/0 (socket says 172.21.15.37:34462) 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 -- 172.21.15.37:0/3437435493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c075590 msgr2=0x7fc90c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c075590 0x7fc90c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 -- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c074c70 msgr2=0x7fc90c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c074c70 0x7fc90c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 -- 172.21.15.37:0/3437435493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc90c1a02f0 con 0x7fc90c10f440 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.862+0000 7fc90a800700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 0x7fc90c19fbb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc90c06bfa0 tx=0x7fc90400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:11.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.863+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc904003a00 con 0x7fc90c10f440 2024-04-26T01:54:11.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.863+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc90400ce50 con 0x7fc90c10f440 2024-04-26T01:54:11.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.863+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc90c1a05e0 con 0x7fc90c10f440 2024-04-26T01:54:11.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.863+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9040046e0 con 0x7fc90c10f440 2024-04-26T01:54:11.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.863+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90c1a7e60 con 0x7fc90c10f440 2024-04-26T01:54:11.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.864+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc90c069730 con 0x7fc90c10f440 2024-04-26T01:54:11.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.866+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc904012050 con 0x7fc90c10f440 2024-04-26T01:54:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.867+0000 7fc903400700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc8f00a3cc0 0x7fc8f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.867+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7fc9040cfb20 con 0x7fc90c10f440 2024-04-26T01:54:11.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.868+0000 7fc909e00700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc8f00a3cc0 0x7fc8f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:11.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.868+0000 7fc909e00700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc8f00a3cc0 0x7fc8f00a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009680 tx=0x7fc8fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:11.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:11.872+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc904022060 con 0x7fc90c10f440 2024-04-26T01:54:12.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:12.159+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fc90c02df80 con 0x7fc90c10f440 2024-04-26T01:54:12.159 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:12 smithi037 ceph-mon[27906]: pgmap v332: 97 pgs: 6 creating+peering, 46 unknown, 45 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:12 smithi045 ceph-mon[32520]: pgmap v332: 97 pgs: 6 creating+peering, 46 unknown, 45 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:12 smithi071 ceph-mon[32578]: pgmap v332: 97 pgs: 6 creating+peering, 46 unknown, 45 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:13.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.021+0000 7fc903400700 1 -- 172.21.15.37:0/3437435493 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fc904010040 con 0x7fc90c10f440 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.026+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc8f00a3cc0 msgr2=0x7fc8f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.026+0000 7fc911200700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc8f00a3cc0 0x7fc8f00a6180 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc009680 tx=0x7fc8fc009260 comp rx=0 tx=0).stop 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.026+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 msgr2=0x7fc90c19fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.026+0000 7fc911200700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 0x7fc90c19fbb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc90c06bfa0 tx=0x7fc90400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.027+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 shutdown_connections 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.027+0000 7fc911200700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc90c074c70 0x7fc90c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.027+0000 7fc911200700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fc8f00a3cc0 0x7fc8f00a6180 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.027+0000 7fc911200700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc90c10f440 0x7fc90c19fbb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.027+0000 7fc911200700 1 --2- 172.21.15.37:0/3437435493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc90c075590 0x7fc90c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:13.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.027+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 >> 172.21.15.37:0/3437435493 conn(0x7fc90c079ba0 msgr2=0x7fc90c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:13.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.028+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 shutdown_connections 2024-04-26T01:54:13.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:13.028+0000 7fc911200700 1 -- 172.21.15.37:0/3437435493 wait complete. 2024-04-26T01:54:13.031 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T01:54:13.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:13 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3437435493' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:54:13.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:13 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:54:13.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:13 smithi071 ceph-mon[32578]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-26T01:54:13.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:13 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3437435493' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:54:13.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:13 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:54:13.446 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:13 smithi037 ceph-mon[27906]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-26T01:54:13.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:13 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3437435493' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:54:13.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:13 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T01:54:13.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:13 smithi045 ceph-mon[32520]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-04-26T01:54:13.779 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:54:14.238 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:14.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:14 smithi037 ceph-mon[27906]: pgmap v333: 97 pgs: 6 creating+peering, 6 unknown, 85 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:14.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:14 smithi037 ceph-mon[27906]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:54:14.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:14 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:54:14.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:14 smithi037 ceph-mon[27906]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-26T01:54:14.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:14 smithi037 ceph-mon[27906]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-26T01:54:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:14 smithi071 ceph-mon[32578]: pgmap v333: 97 pgs: 6 creating+peering, 6 unknown, 85 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:14 smithi071 ceph-mon[32578]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:54:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:14 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:54:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:14 smithi071 ceph-mon[32578]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-26T01:54:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:14 smithi071 ceph-mon[32578]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-26T01:54:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:14 smithi045 ceph-mon[32520]: pgmap v333: 97 pgs: 6 creating+peering, 6 unknown, 85 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:54:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:14 smithi045 ceph-mon[32520]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T01:54:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:14 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T01:54:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:14 smithi045 ceph-mon[32520]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-04-26T01:54:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:14 smithi045 ceph-mon[32520]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-04-26T01:54:15.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:15 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] up:active 2024-04-26T01:54:15.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:15 smithi071 ceph-mon[32578]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:15.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:15 smithi071 ceph-mon[32578]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:15.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:15 smithi071 ceph-mon[32578]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:54:15.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:15 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] up:active 2024-04-26T01:54:15.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:15 smithi037 ceph-mon[27906]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:15.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:15 smithi037 ceph-mon[27906]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:15.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:15 smithi037 ceph-mon[27906]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:54:15.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:15 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] up:active 2024-04-26T01:54:15.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:15 smithi045 ceph-mon[32520]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:15.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:15 smithi045 ceph-mon[32520]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:15.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:15 smithi045 ceph-mon[32520]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T01:54:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:16 smithi037 ceph-mon[27906]: pgmap v335: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-04-26T01:54:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:16 smithi037 ceph-mon[27906]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:54:16.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:16 smithi037 ceph-mon[27906]: Cluster is now healthy 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.401+0000 7fa833e00700 1 -- 172.21.15.37:0/3857888884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c074c70 msgr2=0x7fa82c075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.401+0000 7fa833e00700 1 --2- 172.21.15.37:0/3857888884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c074c70 0x7fa82c075050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa820009990 tx=0x7fa820009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 -- 172.21.15.37:0/3857888884 shutdown_connections 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 --2- 172.21.15.37:0/3857888884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 0x7fa82c111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 --2- 172.21.15.37:0/3857888884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c074c70 0x7fa82c075050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 --2- 172.21.15.37:0/3857888884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c075590 0x7fa82c10ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 -- 172.21.15.37:0/3857888884 >> 172.21.15.37:0/3857888884 conn(0x7fa82c079ba0 msgr2=0x7fa82c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:16.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 -- 172.21.15.37:0/3857888884 shutdown_connections 2024-04-26T01:54:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.402+0000 7fa833e00700 1 -- 172.21.15.37:0/3857888884 wait complete. 2024-04-26T01:54:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:16 smithi071 ceph-mon[32578]: pgmap v335: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-04-26T01:54:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:16 smithi071 ceph-mon[32578]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:54:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:16 smithi071 ceph-mon[32578]: Cluster is now healthy 2024-04-26T01:54:16.402 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 01:54:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: 2024-04-26T01:54:16.170+0000 7f5f5ae84700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 Processor -- start 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 -- start start 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c074c70 0x7fa82c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c075590 0x7fa82c19b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 0x7fa82c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c114ed0 con 0x7fa82c074c70 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c114d50 con 0x7fa82c075590 2024-04-26T01:54:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.404+0000 7fa833e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c115050 con 0x7fa82c10f4a0 2024-04-26T01:54:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 0x7fa82c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c074c70 0x7fa82c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 0x7fa82c19fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34470/0 (socket says 172.21.15.37:34470) 2024-04-26T01:54:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831e00700 1 -- 172.21.15.37:0/4028246879 learned_addr learned my addr 172.21.15.37:0/4028246879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa830a00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c075590 0x7fa82c19b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831400700 1 -- 172.21.15.37:0/4028246879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 msgr2=0x7fa82c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831400700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 0x7fa82c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831400700 1 -- 172.21.15.37:0/4028246879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c075590 msgr2=0x7fa82c19b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831400700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c075590 0x7fa82c19b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa831400700 1 -- 172.21.15.37:0/4028246879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c1a02d0 con 0x7fa82c074c70 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.405+0000 7fa830a00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c075590 0x7fa82c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.406+0000 7fa831400700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c074c70 0x7fa82c19b2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa820000c00 tx=0x7fa8200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.406+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820026070 con 0x7fa82c074c70 2024-04-26T01:54:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.406+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa820021410 con 0x7fa82c074c70 2024-04-26T01:54:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.406+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820017650 con 0x7fa82c074c70 2024-04-26T01:54:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.406+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c1a0560 con 0x7fa82c074c70 2024-04-26T01:54:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.406+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1a7e40 con 0x7fa82c074c70 2024-04-26T01:54:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.407+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c069730 con 0x7fa82c074c70 2024-04-26T01:54:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.410+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa820029070 con 0x7fa82c074c70 2024-04-26T01:54:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.412+0000 7fa82a000700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa8140a3cc0 0x7fa8140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.412+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa8200d1600 con 0x7fa82c074c70 2024-04-26T01:54:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.412+0000 7fa830a00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa8140a3cc0 0x7fa8140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.413+0000 7fa830a00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa8140a3cc0 0x7fa8140a6180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa82c111130 tx=0x7fa818009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.415+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa820030070 con 0x7fa82c074c70 2024-04-26T01:54:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:16 smithi045 ceph-mon[32520]: pgmap v335: 97 pgs: 97 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 1 op/s 2024-04-26T01:54:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:16 smithi045 ceph-mon[32520]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T01:54:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:16 smithi045 ceph-mon[32520]: Cluster is now healthy 2024-04-26T01:54:16.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.650+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa82c1a8010 con 0x7fa82c074c70 2024-04-26T01:54:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.654+0000 7fa82a000700 1 -- 172.21.15.37:0/4028246879 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22123 (secure 0 0 0) 0x7fa820013070 con 0x7fa82c074c70 2024-04-26T01:54:16.654 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:54:16.654 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:12.162599+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa8140a3cc0 msgr2=0x7fa8140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa8140a3cc0 0x7fa8140a6180 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa82c111130 tx=0x7fa818009260 comp rx=0 tx=0).stop 2024-04-26T01:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c074c70 msgr2=0x7fa82c19b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c074c70 0x7fa82c19b2c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa820000c00 tx=0x7fa8200047b0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 shutdown_connections 2024-04-26T01:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa82c074c70 0x7fa82c19b2c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa8140a3cc0 0x7fa8140a6180 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa82c10f4a0 0x7fa82c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 --2- 172.21.15.37:0/4028246879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa82c075590 0x7fa82c19b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.657+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 >> 172.21.15.37:0/4028246879 conn(0x7fa82c079ba0 msgr2=0x7fa82c0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:16.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.658+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 shutdown_connections 2024-04-26T01:54:16.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:16.658+0000 7fa833e00700 1 -- 172.21.15.37:0/4028246879 wait complete. 2024-04-26T01:54:17.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:17 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4028246879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:17.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:17 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4028246879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:17.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:17 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4028246879' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:17.516 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:54:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:18 smithi037 ceph-mon[27906]: pgmap v336: 97 pgs: 97 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T01:54:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:18 smithi071 ceph-mon[32578]: pgmap v336: 97 pgs: 97 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T01:54:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:18 smithi045 ceph-mon[32520]: pgmap v336: 97 pgs: 97 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T01:54:20.178 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:20 smithi037 ceph-mon[27906]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 -- 172.21.15.37:0/2517491160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806b980 msgr2=0x7f818810f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 --2- 172.21.15.37:0/2517491160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806b980 0x7f818810f8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f818000b0a0 tx=0x7f818000b3b0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 -- 172.21.15.37:0/2517491160 shutdown_connections 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 --2- 172.21.15.37:0/2517491160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806b980 0x7f818810f8f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 --2- 172.21.15.37:0/2517491160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8188108a20 0x7f8188108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 --2- 172.21.15.37:0/2517491160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f818806afe0 0x7f818806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 -- 172.21.15.37:0/2517491160 >> 172.21.15.37:0/2517491160 conn(0x7f81880788b0 msgr2=0x7f8188078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 -- 172.21.15.37:0/2517491160 shutdown_connections 2024-04-26T01:54:20.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.212+0000 7f8190600700 1 -- 172.21.15.37:0/2517491160 wait complete. 2024-04-26T01:54:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.213+0000 7f8190600700 1 Processor -- start 2024-04-26T01:54:20.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.213+0000 7f8190600700 1 -- start start 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f8190600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806afe0 0x7f8188101a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f8190600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 0x7f8188101f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f8190600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8188108a20 0x7f8188102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f8190600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112c10 con 0x7f818806afe0 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f8190600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112a90 con 0x7f818806b980 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f8190600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8188112d90 con 0x7f8188108a20 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f818dc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806afe0 0x7f8188101a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f818d200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 0x7f8188101f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f818d200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 0x7f8188101f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:55834/0 (socket says 172.21.15.37:55834) 2024-04-26T01:54:20.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f818d200700 1 -- 172.21.15.37:0/1066168583 learned_addr learned my addr 172.21.15.37:0/1066168583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.214+0000 7f818e600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8188108a20 0x7f8188102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:20.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818dc00700 1 -- 172.21.15.37:0/1066168583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8188108a20 msgr2=0x7f8188102480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818dc00700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8188108a20 0x7f8188102480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818dc00700 1 -- 172.21.15.37:0/1066168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 msgr2=0x7f8188101f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818dc00700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 0x7f8188101f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818dc00700 1 -- 172.21.15.37:0/1066168583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81881a7680 con 0x7f818806afe0 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818d200700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 0x7f8188101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f818dc00700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806afe0 0x7f8188101a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e9f0 tx=0x7f817c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c00c9c0 con 0x7f818806afe0 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817c00cb60 con 0x7f818806afe0 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c018680 con 0x7f818806afe0 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81881a7970 con 0x7f818806afe0 2024-04-26T01:54:20.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.215+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81881a7f00 con 0x7f818806afe0 2024-04-26T01:54:20.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.217+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8188069730 con 0x7f818806afe0 2024-04-26T01:54:20.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.219+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f817c010430 con 0x7f818806afe0 2024-04-26T01:54:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.221+0000 7f8186000700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f81700a3cc0 0x7f81700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.222+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f817c0d0770 con 0x7f818806afe0 2024-04-26T01:54:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.222+0000 7f818d200700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f81700a3cc0 0x7f81700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.222+0000 7f818d200700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f81700a3cc0 0x7f81700a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8188078f20 tx=0x7f8174009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:20.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.225+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f817c09d200 con 0x7f818806afe0 2024-04-26T01:54:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:20 smithi071 ceph-mon[32578]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:54:20.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.463+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f818802e140 con 0x7f818806afe0 2024-04-26T01:54:20.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.466+0000 7f8186000700 1 -- 172.21.15.37:0/1066168583 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22123 (secure 0 0 0) 0x7f817c09cb00 con 0x7f818806afe0 2024-04-26T01:54:20.465 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:54:20.466 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":89,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:12.162599+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:54:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f81700a3cc0 msgr2=0x7f81700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f81700a3cc0 0x7f81700a6180 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8188078f20 tx=0x7f8174009260 comp rx=0 tx=0).stop 2024-04-26T01:54:20.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806afe0 msgr2=0x7f8188101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806afe0 0x7f8188101a00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f817c00e9f0 tx=0x7f817c00ed00 comp rx=0 tx=0).stop 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 shutdown_connections 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f818806afe0 0x7f8188101a00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f81700a3cc0 0x7f81700a6180 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8188108a20 0x7f8188102480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 --2- 172.21.15.37:0/1066168583 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f818806b980 0x7f8188101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.469+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 >> 172.21.15.37:0/1066168583 conn(0x7f81880788b0 msgr2=0x7f81880ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.470+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 shutdown_connections 2024-04-26T01:54:20.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:20.470+0000 7f8190600700 1 -- 172.21.15.37:0/1066168583 wait complete. 2024-04-26T01:54:20.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:20 smithi045 ceph-mon[32520]: pgmap v337: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-26T01:54:21.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:21 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1066168583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:21 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1066168583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:21.331 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-26T01:54:21.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:21 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1066168583' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:54:21.801 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:22.248 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:22 smithi037 ceph-mon[27906]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:54:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:22 smithi071 ceph-mon[32578]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:54:22.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:22 smithi045 ceph-mon[32520]: pgmap v338: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:54:24.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 -- 172.21.15.37:0/3775869892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c104ec0 msgr2=0x7fb45c1052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 --2- 172.21.15.37:0/3775869892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c104ec0 0x7fb45c1052a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb44c0099c0 tx=0x7fb44c009cd0 comp rx=0 tx=0).stop 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 -- 172.21.15.37:0/3775869892 shutdown_connections 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 --2- 172.21.15.37:0/3775869892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c109fa0 0x7fb45c111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 --2- 172.21.15.37:0/3775869892 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c104ec0 0x7fb45c1052a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 --2- 172.21.15.37:0/3775869892 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 -- 172.21.15.37:0/3775869892 >> 172.21.15.37:0/3775869892 conn(0x7fb45c100750 msgr2=0x7fb45c102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 -- 172.21.15.37:0/3775869892 shutdown_connections 2024-04-26T01:54:24.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.115+0000 7fb462200700 1 -- 172.21.15.37:0/3775869892 wait complete. 2024-04-26T01:54:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.116+0000 7fb462200700 1 Processor -- start 2024-04-26T01:54:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.116+0000 7fb462200700 1 -- start start 2024-04-26T01:54:24.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb462200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c104ec0 0x7fb45c19b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb462200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c19b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb462200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c109fa0 0x7fb45c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb462200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c114eb0 con 0x7fb45c104ec0 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb462200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c114d30 con 0x7fb45c105870 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb462200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c115030 con 0x7fb45c109fa0 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c19b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c104ec0 0x7fb45c19b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c109fa0 0x7fb45c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c19b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:55860/0 (socket says 172.21.15.37:55860) 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 -- 172.21.15.37:0/4293951053 learned_addr learned my addr 172.21.15.37:0/4293951053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 -- 172.21.15.37:0/4293951053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c109fa0 msgr2=0x7fb45c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c109fa0 0x7fb45c19fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:24.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 -- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c104ec0 msgr2=0x7fb45c19b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c104ec0 0x7fb45c19b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.117+0000 7fb45aa00700 1 -- 172.21.15.37:0/4293951053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c1a0410 con 0x7fb45c105870 2024-04-26T01:54:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.118+0000 7fb45aa00700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c19b940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e9f0 tx=0x7fb44400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:24.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.118+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444004a30 con 0x7fb45c105870 2024-04-26T01:54:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.118+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c1a0700 con 0x7fb45c105870 2024-04-26T01:54:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.118+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44400cc70 con 0x7fb45c105870 2024-04-26T01:54:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.118+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444005530 con 0x7fb45c105870 2024-04-26T01:54:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.118+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c1a7f80 con 0x7fb45c105870 2024-04-26T01:54:24.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.119+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c069730 con 0x7fb45c105870 2024-04-26T01:54:24.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.121+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb444004bd0 con 0x7fb45c105870 2024-04-26T01:54:24.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.123+0000 7fb453e00700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb4400a3c40 0x7fb4400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.123+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fb4440d0560 con 0x7fb45c105870 2024-04-26T01:54:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.123+0000 7fb45b400700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb4400a3c40 0x7fb4400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:24.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.124+0000 7fb45b400700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb4400a3c40 0x7fb4400a6100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb44c003760 tx=0x7fb44c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:24.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.128+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb44400ce20 con 0x7fb45c105870 2024-04-26T01:54:24.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:24 smithi037 ceph-mon[27906]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:54:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:24 smithi071 ceph-mon[32578]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:54:24.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:24.409+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb45c02df80 con 0x7fb45c105870 2024-04-26T01:54:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:24 smithi045 ceph-mon[32520]: pgmap v339: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T01:54:25.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.078+0000 7fb453e00700 1 -- 172.21.15.37:0/4293951053 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fb44409cff0 con 0x7fb45c105870 2024-04-26T01:54:25.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb4400a3c40 msgr2=0x7fb4400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:25.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb4400a3c40 0x7fb4400a6100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb44c003760 tx=0x7fb44c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:54:25.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 msgr2=0x7fb45c19b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:25.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c19b940 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e9f0 tx=0x7fb44400ed00 comp rx=0 tx=0).stop 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 shutdown_connections 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c104ec0 0x7fb45c19b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb4400a3c40 0x7fb4400a6100 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb45c109fa0 0x7fb45c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 --2- 172.21.15.37:0/4293951053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb45c105870 0x7fb45c19b940 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.083+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 >> 172.21.15.37:0/4293951053 conn(0x7fb45c100750 msgr2=0x7fb45c109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.084+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 shutdown_connections 2024-04-26T01:54:25.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:25.084+0000 7fb462200700 1 -- 172.21.15.37:0/4293951053 wait complete. 2024-04-26T01:54:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:25 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4293951053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:54:25.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:25 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:54:25.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:25 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4293951053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:54:25.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:25 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:54:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:25 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4293951053' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:54:25.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:25 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T01:54:25.967 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-26T01:54:25.967 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-26T01:54:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:26 smithi037 ceph-mon[27906]: pgmap v340: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T01:54:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:26 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:54:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:26 smithi037 ceph-mon[27906]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:26 smithi071 ceph-mon[32578]: pgmap v340: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T01:54:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:26 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:54:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:26 smithi071 ceph-mon[32578]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:26.461 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:26 smithi045 ceph-mon[32520]: pgmap v340: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T01:54:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:26 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T01:54:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:26 smithi045 ceph-mon[32520]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-04-26T01:54:28.299 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:28 smithi037 ceph-mon[27906]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2024-04-26T01:54:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:28 smithi071 ceph-mon[32578]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2024-04-26T01:54:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:28 smithi045 ceph-mon[32520]: pgmap v341: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 3 op/s 2024-04-26T01:54:28.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.845+0000 7fa803e00700 1 -- 172.21.15.37:0/2801471248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc103450 msgr2=0x7fa7fc10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.845+0000 7fa803e00700 1 --2- 172.21.15.37:0/2801471248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc103450 0x7fa7fc10f8b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa7f400b0a0 tx=0x7fa7f400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 -- 172.21.15.37:0/2801471248 shutdown_connections 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 --2- 172.21.15.37:0/2801471248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc103450 0x7fa7fc10f8b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 --2- 172.21.15.37:0/2801471248 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc108a50 0x7fa7fc108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 --2- 172.21.15.37:0/2801471248 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7fc102a90 0x7fa7fc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 -- 172.21.15.37:0/2801471248 >> 172.21.15.37:0/2801471248 conn(0x7fa7fc079ba0 msgr2=0x7fa7fc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 -- 172.21.15.37:0/2801471248 shutdown_connections 2024-04-26T01:54:28.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.846+0000 7fa803e00700 1 -- 172.21.15.37:0/2801471248 wait complete. 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.847+0000 7fa803e00700 1 Processor -- start 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.847+0000 7fa803e00700 1 -- start start 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa803e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 0x7fa7fc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa803e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7fc103450 0x7fa7fc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa803e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc108a50 0x7fa7fc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa803e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc112c50 con 0x7fa7fc108a50 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa803e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc112ad0 con 0x7fa7fc102a90 2024-04-26T01:54:28.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa803e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa7fc112dd0 con 0x7fa7fc103450 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa801400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 0x7fa7fc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa801400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 0x7fa7fc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:55878/0 (socket says 172.21.15.37:55878) 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa801400700 1 -- 172.21.15.37:0/1403387639 learned_addr learned my addr 172.21.15.37:0/1403387639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa801e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc108a50 0x7fa7fc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa800a00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7fc103450 0x7fa7fc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa801400700 1 -- 172.21.15.37:0/1403387639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7fc103450 msgr2=0x7fa7fc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.848+0000 7fa801400700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7fc103450 0x7fa7fc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa801400700 1 -- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc108a50 msgr2=0x7fa7fc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa801400700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc108a50 0x7fa7fc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa801400700 1 -- 172.21.15.37:0/1403387639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7fc1a7650 con 0x7fa7fc102a90 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa801400700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 0x7fa7fc074d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000eb30 tx=0x7fa7f000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f000cbd0 con 0x7fa7fc102a90 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7f00044d0 con 0x7fa7fc102a90 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7fc1a7940 con 0x7fa7fc102a90 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7f0003e30 con 0x7fa7fc102a90 2024-04-26T01:54:28.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.849+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7fc1a7e80 con 0x7fa7fc102a90 2024-04-26T01:54:28.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.850+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7fc069730 con 0x7fa7fc102a90 2024-04-26T01:54:28.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.852+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7f0004000 con 0x7fa7fc102a90 2024-04-26T01:54:28.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.854+0000 7fa7fa000700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa7e40a3c40 0x7fa7e40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:28.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.854+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa7f00d0600 con 0x7fa7fc102a90 2024-04-26T01:54:28.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.854+0000 7fa800a00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa7e40a3c40 0x7fa7e40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:28.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.855+0000 7fa800a00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa7e40a3c40 0x7fa7e40a6100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc07a210 tx=0x7fa7e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:28.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:28.856+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7f00042f0 con 0x7fa7fc102a90 2024-04-26T01:54:29.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.137+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fa7fc02e1a0 con 0x7fa7fc102a90 2024-04-26T01:54:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.531+0000 7fa7fa000700 1 -- 172.21.15.37:0/1403387639 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fa7f009d090 con 0x7fa7fc102a90 2024-04-26T01:54:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa7e40a3c40 msgr2=0x7fa7e40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa7e40a3c40 0x7fa7e40a6100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc07a210 tx=0x7fa7e8009260 comp rx=0 tx=0).stop 2024-04-26T01:54:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 msgr2=0x7fa7fc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 0x7fa7fc074d40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa7f000eb30 tx=0x7fa7f000ee40 comp rx=0 tx=0).stop 2024-04-26T01:54:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 shutdown_connections 2024-04-26T01:54:29.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7fc108a50 0x7fa7fc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:29.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa7e40a3c40 0x7fa7e40a6100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:29.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa7fc102a90 0x7fa7fc074d40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:29.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 --2- 172.21.15.37:0/1403387639 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa7fc103450 0x7fa7fc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:29.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.535+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 >> 172.21.15.37:0/1403387639 conn(0x7fa7fc079ba0 msgr2=0x7fa7fc101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:29.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.536+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 shutdown_connections 2024-04-26T01:54:29.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:29.536+0000 7fa803e00700 1 -- 172.21.15.37:0/1403387639 wait complete. 2024-04-26T01:54:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:30 smithi037 ceph-mon[27906]: pgmap v342: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T01:54:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:30 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1403387639' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:54:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:30 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:54:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:30 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:54:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:30 smithi037 ceph-mon[27906]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T01:54:30.356 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-26T01:54:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:30 smithi071 ceph-mon[32578]: pgmap v342: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T01:54:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:30 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1403387639' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:54:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:30 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:54:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:30 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:54:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:30 smithi071 ceph-mon[32578]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T01:54:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:30 smithi045 ceph-mon[32520]: pgmap v342: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-26T01:54:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:30 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1403387639' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:54:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:30 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T01:54:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:30 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T01:54:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:30 smithi045 ceph-mon[32520]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T01:54:30.850 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:32 smithi037 ceph-mon[27906]: pgmap v343: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 0 op/s 2024-04-26T01:54:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:32 smithi071 ceph-mon[32578]: pgmap v343: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 0 op/s 2024-04-26T01:54:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:32 smithi045 ceph-mon[32520]: pgmap v343: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s rd, 0 op/s 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 -- 172.21.15.37:0/1179091095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 msgr2=0x7f467c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 --2- 172.21.15.37:0/1179091095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 0x7f467c108e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f466c009990 tx=0x7f466c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 -- 172.21.15.37:0/1179091095 shutdown_connections 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 --2- 172.21.15.37:0/1179091095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c10f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 --2- 172.21.15.37:0/1179091095 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 0x7f467c108e00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 --2- 172.21.15.37:0/1179091095 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f467c06afe0 0x7f467c06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 -- 172.21.15.37:0/1179091095 >> 172.21.15.37:0/1179091095 conn(0x7f467c0788b0 msgr2=0x7f467c078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:33.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 -- 172.21.15.37:0/1179091095 shutdown_connections 2024-04-26T01:54:33.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.102+0000 7f4682e00700 1 -- 172.21.15.37:0/1179091095 wait complete. 2024-04-26T01:54:33.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.104+0000 7f4682e00700 1 Processor -- start 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.104+0000 7f4682e00700 1 -- start start 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.104+0000 7f4682e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f467c06afe0 0x7f467c19b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.104+0000 7f4682e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c19b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.104+0000 7f4682e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 0x7f467c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4682e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112c10 con 0x7f467c06b980 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4682e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112a90 con 0x7f467c06afe0 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4682e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f467c112d90 con 0x7f467c108a20 2024-04-26T01:54:33.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c19b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39546/0 (socket says 172.21.15.37:39546) 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 -- 172.21.15.37:0/2627572704 learned_addr learned my addr 172.21.15.37:0/2627572704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f467be00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f467c06afe0 0x7f467c19b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4680e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 0x7f467c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 -- 172.21.15.37:0/2627572704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 msgr2=0x7f467c19fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 0x7f467c19fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:33.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 -- 172.21.15.37:0/2627572704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f467c06afe0 msgr2=0x7f467c19b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f467c06afe0 0x7f467c19b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.105+0000 7f4673e00700 1 -- 172.21.15.37:0/2627572704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c1a0450 con 0x7f467c06b980 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.106+0000 7f4673e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c19b980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f466400e9c0 tx=0x7f466400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.106+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466400cbd0 con 0x7f467c06b980 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.106+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f466400ee60 con 0x7f467c06b980 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.106+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c1a0740 con 0x7f467c06b980 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.106+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46640055c0 con 0x7f467c06b980 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.106+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c1a7fc0 con 0x7f467c06b980 2024-04-26T01:54:33.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.108+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467c069730 con 0x7f467c06b980 2024-04-26T01:54:33.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.111+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4664004750 con 0x7f467c06b980 2024-04-26T01:54:33.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.113+0000 7f4679600700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f46600a3cc0 0x7f46600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:33.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.113+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f46640d0790 con 0x7f467c06b980 2024-04-26T01:54:33.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.113+0000 7f467be00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f46600a3cc0 0x7f46600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:33.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.114+0000 7f467be00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f46600a3cc0 0x7f46600a6180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f466c006010 tx=0x7f466c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:33.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.116+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4664053ff0 con 0x7f467c06b980 2024-04-26T01:54:33.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:33.401+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f467c02df80 con 0x7f467c06b980 2024-04-26T01:54:34.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.108+0000 7f4679600700 1 -- 172.21.15.37:0/2627572704 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v19) ==== 115+0+0 (secure 0 0 0) 0x7f466409d220 con 0x7f467c06b980 2024-04-26T01:54:34.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f46600a3cc0 msgr2=0x7f46600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:34.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f46600a3cc0 0x7f46600a6180 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f466c006010 tx=0x7f466c00b260 comp rx=0 tx=0).stop 2024-04-26T01:54:34.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 msgr2=0x7f467c19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:34.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c19b980 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f466400e9c0 tx=0x7f466400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:54:34.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 shutdown_connections 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f46600a3cc0 0x7f46600a6180 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f467c108a20 0x7f467c19fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f467c06afe0 0x7f467c19b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 --2- 172.21.15.37:0/2627572704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f467c06b980 0x7f467c19b980 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 >> 172.21.15.37:0/2627572704 conn(0x7f467c0788b0 msgr2=0x7f467c10dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 shutdown_connections 2024-04-26T01:54:34.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:34.113+0000 7f4682e00700 1 -- 172.21.15.37:0/2627572704 wait complete. 2024-04-26T01:54:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:34 smithi071 ceph-mon[32578]: pgmap v344: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 0 op/s 2024-04-26T01:54:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:34 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2627572704' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:54:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:34 smithi037 ceph-mon[27906]: pgmap v344: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 0 op/s 2024-04-26T01:54:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:34 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2627572704' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:54:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:34 smithi045 ceph-mon[32520]: pgmap v344: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 0 op/s 2024-04-26T01:54:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:34 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2627572704' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T01:54:34.931 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T01:54:34.931 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-26T01:54:35.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:35 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2627572704' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:54:35.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:35 smithi037 ceph-mon[27906]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T01:54:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:35 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2627572704' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:54:35.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:35 smithi071 ceph-mon[32578]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T01:54:35.402 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:35.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:35 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2627572704' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T01:54:35.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:35 smithi045 ceph-mon[32520]: fsmap cephfs:1 {0=i=up:active} 1 up:standby-replay 10 up:standby 2024-04-26T01:54:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:36 smithi071 ceph-mon[32578]: pgmap v345: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T01:54:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:36 smithi037 ceph-mon[27906]: pgmap v345: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T01:54:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:36 smithi045 ceph-mon[32520]: pgmap v345: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 -- 172.21.15.37:0/354804841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c10f440 msgr2=0x7fd06c111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 --2- 172.21.15.37:0/354804841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c10f440 0x7fd06c111830 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd06400b0a0 tx=0x7fd06400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 -- 172.21.15.37:0/354804841 shutdown_connections 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 --2- 172.21.15.37:0/354804841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c10f440 0x7fd06c111830 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 --2- 172.21.15.37:0/354804841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c074c70 0x7fd06c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 --2- 172.21.15.37:0/354804841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c075590 0x7fd06c10ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 -- 172.21.15.37:0/354804841 >> 172.21.15.37:0/354804841 conn(0x7fd06c079ba0 msgr2=0x7fd06c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 -- 172.21.15.37:0/354804841 shutdown_connections 2024-04-26T01:54:37.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.709+0000 7fd073400700 1 -- 172.21.15.37:0/354804841 wait complete. 2024-04-26T01:54:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 Processor -- start 2024-04-26T01:54:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 -- start start 2024-04-26T01:54:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 0x7fd06c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:37.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c075590 0x7fd06c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 0x7fd06c19fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c114e40 con 0x7fd06c10f440 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c114cc0 con 0x7fd06c074c70 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.711+0000 7fd073400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd06c114fc0 con 0x7fd06c075590 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 0x7fd06c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd071400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 0x7fd06c19fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd071400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 0x7fd06c19fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39564/0 (socket says 172.21.15.37:39564) 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 0x7fd06c19b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:38258/0 (socket says 172.21.15.37:38258) 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd071400700 1 -- 172.21.15.37:0/2957714954 learned_addr learned my addr 172.21.15.37:0/2957714954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd063e00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c075590 0x7fd06c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 -- 172.21.15.37:0/2957714954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c075590 msgr2=0x7fd06c19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c075590 0x7fd06c19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 -- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 msgr2=0x7fd06c19fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:37.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 0x7fd06c19fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:37.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd070a00700 1 -- 172.21.15.37:0/2957714954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06c1a02a0 con 0x7fd06c074c70 2024-04-26T01:54:37.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd063e00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c075590 0x7fd06c19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:37.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.712+0000 7fd071400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 0x7fd06c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:37.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.713+0000 7fd070a00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 0x7fd06c19b290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00e9f0 tx=0x7fd05c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.713+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c004220 con 0x7fd06c074c70 2024-04-26T01:54:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.713+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd05c0043c0 con 0x7fd06c074c70 2024-04-26T01:54:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.713+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd06c1a0590 con 0x7fd06c074c70 2024-04-26T01:54:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.713+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05c005020 con 0x7fd06c074c70 2024-04-26T01:54:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.713+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd06c1a7e10 con 0x7fd06c074c70 2024-04-26T01:54:37.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.714+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd06c069730 con 0x7fd06c074c70 2024-04-26T01:54:37.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.716+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd05c004750 con 0x7fd06c074c70 2024-04-26T01:54:37.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.717+0000 7fd06a000700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0500a3c40 0x7fd0500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.718+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fd05c0d02d0 con 0x7fd06c074c70 2024-04-26T01:54:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.718+0000 7fd063e00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0500a3c40 0x7fd0500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:37.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.718+0000 7fd063e00700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0500a3c40 0x7fd0500a6100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd06c111070 tx=0x7fd054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.722+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd05c09cd60 con 0x7fd06c074c70 2024-04-26T01:54:37.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:37.989+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fd06c100320 con 0x7fd0500a3c40 2024-04-26T01:54:38.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.018+0000 7fd06a000700 1 -- 172.21.15.37:0/2957714954 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd06c100320 con 0x7fd0500a3c40 2024-04-26T01:54:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0500a3c40 msgr2=0x7fd0500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0500a3c40 0x7fd0500a6100 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd06c111070 tx=0x7fd054009260 comp rx=0 tx=0).stop 2024-04-26T01:54:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 msgr2=0x7fd06c19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 0x7fd06c19b290 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd05c00e9f0 tx=0x7fd05c00ed00 comp rx=0 tx=0).stop 2024-04-26T01:54:38.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 shutdown_connections 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd06c10f440 0x7fd06c19fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0500a3c40 0x7fd0500a6100 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd06c074c70 0x7fd06c19b290 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 --2- 172.21.15.37:0/2957714954 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd06c075590 0x7fd06c19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 >> 172.21.15.37:0/2957714954 conn(0x7fd06c079ba0 msgr2=0x7fd06c0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.021+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 shutdown_connections 2024-04-26T01:54:38.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:38.022+0000 7fd073400700 1 -- 172.21.15.37:0/2957714954 wait complete. 2024-04-26T01:54:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:38 smithi071 ceph-mon[32578]: pgmap v346: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T01:54:38.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:38 smithi037 ceph-mon[27906]: pgmap v346: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T01:54:38.441 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:54:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:54:37.998+0000 7f641b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:38.441 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:54:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:54:37.998+0000 7f641b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:38.441 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:54:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:54:37.998+0000 7f641b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:38.441 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:54:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:54:37.998+0000 7f641b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:38.441 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:54:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:54:37.998+0000 7f641b000700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:54:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:38 smithi045 ceph-mon[32520]: pgmap v346: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-04-26T01:54:38.898 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T01:54:39.360 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:39.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:39 smithi071 ceph-mon[32578]: from='client.24575 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:39.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:39 smithi037 ceph-mon[27906]: from='client.24575 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:39.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:39 smithi045 ceph-mon[32520]: from='client.24575 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:40.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:40 smithi037 ceph-mon[27906]: pgmap v347: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T01:54:40.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:40 smithi037 ceph-mon[27906]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T01:54:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:40 smithi071 ceph-mon[32578]: pgmap v347: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T01:54:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:40 smithi071 ceph-mon[32578]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T01:54:40.405 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:40 smithi045 ceph-mon[32520]: pgmap v347: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T01:54:40.405 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:40 smithi045 ceph-mon[32520]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-04-26T01:54:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 -- 172.21.15.37:0/2813323492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8102330 msgr2=0x7f15e810e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 --2- 172.21.15.37:0/2813323492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8102330 0x7f15e810e820 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f15e00099c0 tx=0x7f15e0009cd0 comp rx=0 tx=0).stop 2024-04-26T01:54:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 -- 172.21.15.37:0/2813323492 shutdown_connections 2024-04-26T01:54:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 --2- 172.21.15.37:0/2813323492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8102330 0x7f15e810e820 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 --2- 172.21.15.37:0/2813323492 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e81079c0 0x7f15e8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 --2- 172.21.15.37:0/2813323492 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e8101970 0x7f15e8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 -- 172.21.15.37:0/2813323492 >> 172.21.15.37:0/2813323492 conn(0x7f15e8078980 msgr2=0x7f15e8078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 -- 172.21.15.37:0/2813323492 shutdown_connections 2024-04-26T01:54:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.762+0000 7f15ee000700 1 -- 172.21.15.37:0/2813323492 wait complete. 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.764+0000 7f15ee000700 1 Processor -- start 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.764+0000 7f15ee000700 1 -- start start 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15ee000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8101970 0x7f15e8103520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15ee000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 0x7f15e8103a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15ee000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e81079c0 0x7f15e8103fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15ee000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8111b30 con 0x7f15e8101970 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15ee000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81119b0 con 0x7f15e81079c0 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15ee000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8111cb0 con 0x7f15e8102330 2024-04-26T01:54:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15e7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8101970 0x7f15e8103520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15e6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 0x7f15e8103a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15e7400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8101970 0x7f15e8103520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42730/0 (socket says 172.21.15.37:42730) 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15e6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 0x7f15e8103a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37010/0 (socket says 172.21.15.37:37010) 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15e7400700 1 -- 172.21.15.37:0/595789599 learned_addr learned my addr 172.21.15.37:0/595789599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.765+0000 7f15e7e00700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e81079c0 0x7f15e8103fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15e6a00700 1 -- 172.21.15.37:0/595789599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e81079c0 msgr2=0x7f15e8103fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15e6a00700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e81079c0 0x7f15e8103fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15e6a00700 1 -- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8101970 msgr2=0x7f15e8103520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15e6a00700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8101970 0x7f15e8103520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15e6a00700 1 -- 172.21.15.37:0/595789599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e81a76a0 con 0x7f15e8102330 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15e6a00700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 0x7f15e8103a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15d400e9f0 tx=0x7f15d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d400c9c0 con 0x7f15e8102330 2024-04-26T01:54:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e81a7990 con 0x7f15e8102330 2024-04-26T01:54:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d400cb60 con 0x7f15e8102330 2024-04-26T01:54:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4018680 con 0x7f15e8102330 2024-04-26T01:54:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.766+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81a7ed0 con 0x7f15e8102330 2024-04-26T01:54:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.768+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e8069730 con 0x7f15e8102330 2024-04-26T01:54:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.769+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f15d4010430 con 0x7f15e8102330 2024-04-26T01:54:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.771+0000 7f15dfe00700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f15cc0a3cd0 0x7f15cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.771+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f15d40d05f0 con 0x7f15e8102330 2024-04-26T01:54:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.771+0000 7f15e7400700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f15cc0a3cd0 0x7f15cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:41.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.772+0000 7f15e7400700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f15cc0a3cd0 0x7f15cc0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15d80094f0 tx=0x7f15d8009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:41.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:41.776+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f15d409cf20 con 0x7f15e8102330 2024-04-26T01:54:42.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.041+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f15e8062e80 con 0x7f15cc0a3cd0 2024-04-26T01:54:42.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.080+0000 7f15dfe00700 1 -- 172.21.15.37:0/595789599 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f15e8062e80 con 0x7f15cc0a3cd0 2024-04-26T01:54:42.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f15cc0a3cd0 msgr2=0x7f15cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f15cc0a3cd0 0x7f15cc0a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15d80094f0 tx=0x7f15d8009040 comp rx=0 tx=0).stop 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 msgr2=0x7f15e8103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 0x7f15e8103a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f15d400e9f0 tx=0x7f15d400ed00 comp rx=0 tx=0).stop 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 shutdown_connections 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15e8101970 0x7f15e8103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f15cc0a3cd0 0x7f15cc0a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f15e81079c0 0x7f15e8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 --2- 172.21.15.37:0/595789599 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f15e8102330 0x7f15e8103a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 >> 172.21.15.37:0/595789599 conn(0x7f15e8078980 msgr2=0x7f15e806b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.084+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 shutdown_connections 2024-04-26T01:54:42.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:42.085+0000 7f15ee000700 1 -- 172.21.15.37:0/595789599 wait complete. 2024-04-26T01:54:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:42 smithi071 ceph-mon[32578]: pgmap v348: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T01:54:42.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:42 smithi037 ceph-mon[27906]: pgmap v348: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T01:54:42.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:42 smithi045 ceph-mon[32520]: pgmap v348: 97 pgs: 97 active+clean; 579 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T01:54:43.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:54:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:43 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T01:54:43.495 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-26T01:54:43.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-26T01:54:43.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:54:43.506 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:43 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='client.24622 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]: dispatch 2024-04-26T01:54:43.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:43 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-04-26T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: pgmap v349: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-04-26T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-26T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-26T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-04-26T01:54:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-26T01:54:44.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-26T01:54:44.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:54:44.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T01:54:44.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:44 smithi037 ceph-mon[27906]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: pgmap v349: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:54:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T01:54:44.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:44 smithi045 ceph-mon[32520]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:54:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: pgmap v349: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 2.1 KiB/s wr, 3 op/s 2024-04-26T01:54:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2024-04-26T01:54:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-04-26T01:54:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-04-26T01:54:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 7]}]': finished 2024-04-26T01:54:44.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.b", "id": [3, 9]}]': finished 2024-04-26T01:54:44.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.20", "id": [5, 9]}]': finished 2024-04-26T01:54:44.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [4, 8]}]': finished 2024-04-26T01:54:44.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-04-26T01:54:44.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:44 smithi071 ceph-mon[32578]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T01:54:45.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:45 smithi037 ceph-mon[27906]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:54:45.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:45 smithi045 ceph-mon[32520]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:54:45.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:45 smithi071 ceph-mon[32578]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T01:54:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 -- 172.21.15.37:0/3494234231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 msgr2=0x7f61dc115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 --2- 172.21.15.37:0/3494234231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc115f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009f90 tx=0x7f61d400b330 comp rx=0 tx=0).stop 2024-04-26T01:54:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 -- 172.21.15.37:0/3494234231 shutdown_connections 2024-04-26T01:54:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 --2- 172.21.15.37:0/3494234231 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc1092d0 0x7f61dc1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 --2- 172.21.15.37:0/3494234231 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc115f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 --2- 172.21.15.37:0/3494234231 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc109c80 0x7f61dc10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 -- 172.21.15.37:0/3494234231 >> 172.21.15.37:0/3494234231 conn(0x7f61dc076210 msgr2=0x7f61dc107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:45.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 -- 172.21.15.37:0/3494234231 shutdown_connections 2024-04-26T01:54:45.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.810+0000 7f61e1800700 1 -- 172.21.15.37:0/3494234231 wait complete. 2024-04-26T01:54:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.811+0000 7f61e1800700 1 Processor -- start 2024-04-26T01:54:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 -- start start 2024-04-26T01:54:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc1092d0 0x7f61dc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc109c80 0x7f61dc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc1192a0 con 0x7f61dc109c80 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc119120 con 0x7f61dc10e3b0 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61e1800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f61dc119420 con 0x7f61dc1092d0 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61da000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc109c80 0x7f61dc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.812+0000 7f61daa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc1092d0 0x7f61dc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61daa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc1092d0 0x7f61dc19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37028/0 (socket says 172.21.15.37:37028) 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61da000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc109c80 0x7f61dc19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42752/0 (socket says 172.21.15.37:42752) 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61daa00700 1 -- 172.21.15.37:0/4117258463 learned_addr learned my addr 172.21.15.37:0/4117258463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 -- 172.21.15.37:0/4117258463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc1092d0 msgr2=0x7f61dc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc1092d0 0x7f61dc19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 -- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc109c80 msgr2=0x7f61dc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc109c80 0x7f61dc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:45.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 -- 172.21.15.37:0/4117258463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61dc1a4710 con 0x7f61dc10e3b0 2024-04-26T01:54:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.813+0000 7f61db400700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc1a3fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b6c0 tx=0x7f61d400cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.814+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4021070 con 0x7f61dc10e3b0 2024-04-26T01:54:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.814+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4007d60 con 0x7f61dc10e3b0 2024-04-26T01:54:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.814+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4005040 con 0x7f61dc10e3b0 2024-04-26T01:54:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.814+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61dc1a49a0 con 0x7f61dc10e3b0 2024-04-26T01:54:45.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.814+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61dc1ac280 con 0x7f61dc10e3b0 2024-04-26T01:54:45.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.815+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61dc069730 con 0x7f61dc10e3b0 2024-04-26T01:54:45.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.818+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f61d4024070 con 0x7f61dc10e3b0 2024-04-26T01:54:45.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.820+0000 7f61d3400700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61c00a3d50 0x7f61c00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:45.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.820+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f61d400f020 con 0x7f61dc10e3b0 2024-04-26T01:54:45.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.820+0000 7f61daa00700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61c00a3d50 0x7f61c00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:45.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.821+0000 7f61daa00700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61c00a3d50 0x7f61c00a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61cc005d10 tx=0x7f61cc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:45.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:45.821+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61d402b020 con 0x7f61dc10e3b0 2024-04-26T01:54:46.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.095+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f61dc062e80 con 0x7f61c00a3d50 2024-04-26T01:54:46.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.103+0000 7f61d3400700 1 -- 172.21.15.37:0/4117258463 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f61dc062e80 con 0x7f61c00a3d50 2024-04-26T01:54:46.101 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/137ce573-3792-4df1-a81f-ce4b3773d996 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61c00a3d50 msgr2=0x7f61c00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61c00a3d50 0x7f61c00a6210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f61cc005d10 tx=0x7f61cc009310 comp rx=0 tx=0).stop 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 msgr2=0x7f61dc1a3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc1a3fd0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f61d400b6c0 tx=0x7f61d400cc30 comp rx=0 tx=0).stop 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 shutdown_connections 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f61c00a3d50 0x7f61c00a6210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f61dc10e3b0 0x7f61dc1a3fd0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f61dc1092d0 0x7f61dc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 --2- 172.21.15.37:0/4117258463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61dc109c80 0x7f61dc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 >> 172.21.15.37:0/4117258463 conn(0x7f61dc076210 msgr2=0x7f61dc0767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 shutdown_connections 2024-04-26T01:54:46.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:46.106+0000 7f61e1800700 1 -- 172.21.15.37:0/4117258463 wait complete. 2024-04-26T01:54:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:46 smithi037 ceph-mon[27906]: pgmap v352: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-26T01:54:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:46 smithi045 ceph-mon[32520]: pgmap v352: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-26T01:54:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:46 smithi071 ceph-mon[32578]: pgmap v352: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 3.2 KiB/s wr, 4 op/s 2024-04-26T01:54:46.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T01:54:47.439 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:47.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:47 smithi037 ceph-mon[27906]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:47.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:47 smithi045 ceph-mon[32520]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:47.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:47 smithi071 ceph-mon[32578]: from='client.24593 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:48.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:48 smithi037 ceph-mon[27906]: pgmap v353: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.1 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T01:54:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:48 smithi045 ceph-mon[32520]: pgmap v353: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.1 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T01:54:48.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:48 smithi071 ceph-mon[32578]: pgmap v353: 97 pgs: 97 active+clean; 599 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 3.1 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-04-26T01:54:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.745+0000 7ffb42600700 1 -- 172.21.15.37:0/417357368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c108a30 msgr2=0x7ffb3c108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.745+0000 7ffb42600700 1 --2- 172.21.15.37:0/417357368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c108a30 0x7ffb3c108e10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c0098f0 tx=0x7ffb2c009c00 comp rx=0 tx=0).stop 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 -- 172.21.15.37:0/417357368 shutdown_connections 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 --2- 172.21.15.37:0/417357368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06b0f0 0x7ffb3c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 --2- 172.21.15.37:0/417357368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c108a30 0x7ffb3c108e10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 --2- 172.21.15.37:0/417357368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c06a750 0x7ffb3c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 -- 172.21.15.37:0/417357368 >> 172.21.15.37:0/417357368 conn(0x7ffb3c0788f0 msgr2=0x7ffb3c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 -- 172.21.15.37:0/417357368 shutdown_connections 2024-04-26T01:54:49.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.746+0000 7ffb42600700 1 -- 172.21.15.37:0/417357368 wait complete. 2024-04-26T01:54:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.747+0000 7ffb42600700 1 Processor -- start 2024-04-26T01:54:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 -- start start 2024-04-26T01:54:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 0x7ffb3c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c06b0f0 0x7ffb3c19b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:49.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c108a30 0x7ffb3c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c112c20 con 0x7ffb3c108a30 2024-04-26T01:54:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c112aa0 con 0x7ffb3c06b0f0 2024-04-26T01:54:49.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.748+0000 7ffb42600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffb3c112da0 con 0x7ffb3c06a750 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c108a30 0x7ffb3c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 0x7ffb3c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 0x7ffb3c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37042/0 (socket says 172.21.15.37:37042) 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 -- 172.21.15.37:0/2590837039 learned_addr learned my addr 172.21.15.37:0/2590837039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3be00700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c108a30 0x7ffb3c19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42772/0 (socket says 172.21.15.37:42772) 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3aa00700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c06b0f0 0x7ffb3c19b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 -- 172.21.15.37:0/2590837039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c06b0f0 msgr2=0x7ffb3c19b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c06b0f0 0x7ffb3c19b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 -- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c108a30 msgr2=0x7ffb3c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c108a30 0x7ffb3c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 -- 172.21.15.37:0/2590837039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb3c1a0380 con 0x7ffb3c06a750 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.749+0000 7ffb3b400700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 0x7ffb3c19b370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009230 tx=0x7ffb2c00fa70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.750+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0050c0 con 0x7ffb3c06a750 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.750+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb2c00b560 con 0x7ffb3c06a750 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.750+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb3c1a0610 con 0x7ffb3c06a750 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.750+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c003a00 con 0x7ffb3c06a750 2024-04-26T01:54:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.750+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb3c1a7ef0 con 0x7ffb3c06a750 2024-04-26T01:54:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.752+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3c104170 con 0x7ffb3c06a750 2024-04-26T01:54:49.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.754+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb2c00b710 con 0x7ffb3c06a750 2024-04-26T01:54:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.755+0000 7ffb33e00700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ffb200a3cd0 0x7ffb200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.755+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7ffb2c0d14c0 con 0x7ffb3c06a750 2024-04-26T01:54:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.755+0000 7ffb3aa00700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ffb200a3cd0 0x7ffb200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.756+0000 7ffb3aa00700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ffb200a3cd0 0x7ffb200a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009750 tx=0x7ffb28009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:49.760+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb2c013030 con 0x7ffb3c06a750 2024-04-26T01:54:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.028+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ffb3c19c6d0 con 0x7ffb200a3cd0 2024-04-26T01:54:50.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.066+0000 7ffb33e00700 1 -- 172.21.15.37:0/2590837039 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffb3c19c6d0 con 0x7ffb200a3cd0 2024-04-26T01:54:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ffb200a3cd0 msgr2=0x7ffb200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ffb200a3cd0 0x7ffb200a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffb28009750 tx=0x7ffb28009310 comp rx=0 tx=0).stop 2024-04-26T01:54:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 msgr2=0x7ffb3c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 0x7ffb3c19b370 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c009230 tx=0x7ffb2c00fa70 comp rx=0 tx=0).stop 2024-04-26T01:54:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 shutdown_connections 2024-04-26T01:54:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb3c108a30 0x7ffb3c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ffb200a3cd0 0x7ffb200a6190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb3c06a750 0x7ffb3c19b370 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 --2- 172.21.15.37:0/2590837039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb3c06b0f0 0x7ffb3c19b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 >> 172.21.15.37:0/2590837039 conn(0x7ffb3c0788f0 msgr2=0x7ffb3c10df50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 shutdown_connections 2024-04-26T01:54:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:50.069+0000 7ffb42600700 1 -- 172.21.15.37:0/2590837039 wait complete. 2024-04-26T01:54:50.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:50 smithi037 ceph-mon[27906]: pgmap v354: 97 pgs: 97 active+clean; 605 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:50 smithi045 ceph-mon[32520]: pgmap v354: 97 pgs: 97 active+clean; 605 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:50 smithi071 ceph-mon[32578]: pgmap v354: 97 pgs: 97 active+clean; 605 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.1 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:51.071 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T01:54:51.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:51 smithi037 ceph-mon[27906]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:51.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:51 smithi045 ceph-mon[32520]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:51.552 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:51.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:51 smithi071 ceph-mon[32578]: from='client.24634 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:52 smithi037 ceph-mon[27906]: pgmap v355: 97 pgs: 97 active+clean; 605 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:52 smithi045 ceph-mon[32520]: pgmap v355: 97 pgs: 97 active+clean; 605 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:52.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:52 smithi071 ceph-mon[32578]: pgmap v355: 97 pgs: 97 active+clean; 605 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.1 KiB/s wr, 2 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 -- 172.21.15.37:0/1770310427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 msgr2=0x7f096810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 --2- 172.21.15.37:0/1770310427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096810dcd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0954009990 tx=0x7f0954009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 -- 172.21.15.37:0/1770310427 shutdown_connections 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 --2- 172.21.15.37:0/1770310427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09681092d0 0x7f09681096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 --2- 172.21.15.37:0/1770310427 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f096810e3b0 0x7f0968115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 --2- 172.21.15.37:0/1770310427 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096810dcd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 -- 172.21.15.37:0/1770310427 >> 172.21.15.37:0/1770310427 conn(0x7f0968076210 msgr2=0x7f0968107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 -- 172.21.15.37:0/1770310427 shutdown_connections 2024-04-26T01:54:53.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.811+0000 7f0970400700 1 -- 172.21.15.37:0/1770310427 wait complete. 2024-04-26T01:54:53.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.812+0000 7f0970400700 1 Processor -- start 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 -- start start 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09681092d0 0x7f096819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f096810e3b0 0x7f09681a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0968119290 con 0x7f096810e3b0 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0968119110 con 0x7f0968109c80 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f0970400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0968119410 con 0x7f09681092d0 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f096da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09681092d0 0x7f096819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f096da00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09681092d0 0x7f096819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43192/0 (socket says 172.21.15.37:43192) 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f096da00700 1 -- 172.21.15.37:0/2147671213 learned_addr learned my addr 172.21.15.37:0/2147671213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.813+0000 7f096d000700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:53.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096e400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f096810e3b0 0x7f09681a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096d000700 1 -- 172.21.15.37:0/2147671213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09681092d0 msgr2=0x7f096819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096d000700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09681092d0 0x7f096819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096d000700 1 -- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f096810e3b0 msgr2=0x7f09681a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096d000700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f096810e3b0 0x7f09681a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096d000700 1 -- 172.21.15.37:0/2147671213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681a4730 con 0x7f0968109c80 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096e400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f096810e3b0 0x7f09681a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.814+0000 7f096d000700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096819fc60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0954000c00 tx=0x7f0954009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.815+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954026070 con 0x7f0968109c80 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.815+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f095400b610 con 0x7f0968109c80 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.815+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681a49c0 con 0x7f0968109c80 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.815+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0954021e00 con 0x7f0968109c80 2024-04-26T01:54:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.815+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681ac2a0 con 0x7f0968109c80 2024-04-26T01:54:53.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.816+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0968069730 con 0x7f0968109c80 2024-04-26T01:54:53.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.818+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0954029070 con 0x7f0968109c80 2024-04-26T01:54:53.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.820+0000 7f0966000700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f09500a3d50 0x7f09500a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.820+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f09540d11e0 con 0x7f0968109c80 2024-04-26T01:54:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.820+0000 7f096da00700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f09500a3d50 0x7f09500a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:53.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.821+0000 7f096da00700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f09500a3d50 0x7f09500a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f095c009d20 tx=0x7f095c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:53.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:53.824+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f095405dff0 con 0x7f0968109c80 2024-04-26T01:54:54.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.098+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f09681a0a80 con 0x7f09500a3d50 2024-04-26T01:54:54.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.107+0000 7f0966000700 1 -- 172.21.15.37:0/2147671213 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f09681a0a80 con 0x7f09500a3d50 2024-04-26T01:54:54.106 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/ebf00d11-c3a6-4b13-963a-0c3a72a5ed73 2024-04-26T01:54:54.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f09500a3d50 msgr2=0x7f09500a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f09500a3d50 0x7f09500a6210 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f095c009d20 tx=0x7f095c009260 comp rx=0 tx=0).stop 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 msgr2=0x7f096819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096819fc60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0954000c00 tx=0x7f0954009580 comp rx=0 tx=0).stop 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 shutdown_connections 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f096810e3b0 0x7f09681a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f09500a3d50 0x7f09500a6210 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f09681092d0 0x7f096819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 --2- 172.21.15.37:0/2147671213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0968109c80 0x7f096819fc60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.110+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 >> 172.21.15.37:0/2147671213 conn(0x7f0968076210 msgr2=0x7f0968076830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.111+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 shutdown_connections 2024-04-26T01:54:54.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:54.111+0000 7f0970400700 1 -- 172.21.15.37:0/2147671213 wait complete. 2024-04-26T01:54:54.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:54 smithi037 ceph-mon[27906]: pgmap v356: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.7 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:54 smithi045 ceph-mon[32520]: pgmap v356: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.7 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:54.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:54 smithi071 ceph-mon[32578]: pgmap v356: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.7 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:55.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:54:55.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:55 smithi037 ceph-mon[27906]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:55.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:55 smithi045 ceph-mon[32520]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:55.561 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:54:55.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:55 smithi071 ceph-mon[32578]: from='client.24605 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:54:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:56 smithi037 ceph-mon[27906]: pgmap v357: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.5 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:56 smithi045 ceph-mon[32520]: pgmap v357: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.5 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:56.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:56 smithi071 ceph-mon[32578]: pgmap v357: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.5 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 -- 172.21.15.37:0/625705099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 msgr2=0x7fe188102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 --2- 172.21.15.37:0/625705099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe188102f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe174009990 tx=0x7fe174009ca0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 -- 172.21.15.37:0/625705099 shutdown_connections 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 --2- 172.21.15.37:0/625705099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe188108a10 0x7fe188108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 --2- 172.21.15.37:0/625705099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe1881034c0 0x7fe18810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 --2- 172.21.15.37:0/625705099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe188102f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.041+0000 7fe18f000700 1 -- 172.21.15.37:0/625705099 >> 172.21.15.37:0/625705099 conn(0x7fe188079c00 msgr2=0x7fe18807a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.042+0000 7fe18f000700 1 -- 172.21.15.37:0/625705099 shutdown_connections 2024-04-26T01:54:58.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.042+0000 7fe18f000700 1 -- 172.21.15.37:0/625705099 wait complete. 2024-04-26T01:54:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 Processor -- start 2024-04-26T01:54:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 -- start start 2024-04-26T01:54:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe18819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:58.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1881034c0 0x7fe18819b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe188108a10 0x7fe18819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:58.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112c40 con 0x7fe1881034c0 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112ac0 con 0x7fe188108a10 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.043+0000 7fe18f000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe188112dc0 con 0x7fe188102b00 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe18819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1881034c0 0x7fe18819b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe18819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43204/0 (socket says 172.21.15.37:43204) 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 -- 172.21.15.37:0/1277517266 learned_addr learned my addr 172.21.15.37:0/1277517266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe18d000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe188108a10 0x7fe18819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 -- 172.21.15.37:0/1277517266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe188108a10 msgr2=0x7fe18819fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe188108a10 0x7fe18819fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 -- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1881034c0 msgr2=0x7fe18819b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1881034c0 0x7fe18819b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187e00700 1 -- 172.21.15.37:0/1277517266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1881a0400 con 0x7fe188102b00 2024-04-26T01:54:58.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe18d000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe188108a10 0x7fe18819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.044+0000 7fe187400700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1881034c0 0x7fe18819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:54:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.045+0000 7fe187e00700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe18819b3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe17800e9a0 tx=0x7fe17800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.045+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17800ca10 con 0x7fe188102b00 2024-04-26T01:54:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.045+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17800cbb0 con 0x7fe188102b00 2024-04-26T01:54:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.045+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1881a06f0 con 0x7fe188102b00 2024-04-26T01:54:58.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.045+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe178018680 con 0x7fe188102b00 2024-04-26T01:54:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.045+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1881a7f70 con 0x7fe188102b00 2024-04-26T01:54:58.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.046+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe188069730 con 0x7fe188102b00 2024-04-26T01:54:58.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.048+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe178010430 con 0x7fe188102b00 2024-04-26T01:54:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.049+0000 7fe184c00700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe16c0a3e20 0x7fe16c0a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:54:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.050+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe1780d05e0 con 0x7fe188102b00 2024-04-26T01:54:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.050+0000 7fe187400700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe16c0a3e20 0x7fe16c0a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:54:58.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.050+0000 7fe187400700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe16c0a3e20 0x7fe16c0a62e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe188076cd0 tx=0x7fe17400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:54:58.052 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.054+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe17809ce10 con 0x7fe188102b00 2024-04-26T01:54:58.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.338+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1881a8140 con 0x7fe188102b00 2024-04-26T01:54:58.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:58 smithi037 ceph-mon[27906]: pgmap v358: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.0 KiB/s wr, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:58.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:58 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:58.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.339+0000 7fe184c00700 1 -- 172.21.15.37:0/1277517266 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fe1780b4e20 con 0x7fe188102b00 2024-04-26T01:54:58.338 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:54:58.338 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe16c0a3e20 msgr2=0x7fe16c0a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe16c0a3e20 0x7fe16c0a62e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe188076cd0 tx=0x7fe17400b260 comp rx=0 tx=0).stop 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 msgr2=0x7fe18819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe18819b3f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe17800e9a0 tx=0x7fe17800ecb0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 shutdown_connections 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe16c0a3e20 0x7fe16c0a62e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe188108a10 0x7fe18819fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe188102b00 0x7fe18819b3f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 --2- 172.21.15.37:0/1277517266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe1881034c0 0x7fe18819b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:54:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 >> 172.21.15.37:0/1277517266 conn(0x7fe188079c00 msgr2=0x7fe18810dca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:54:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 shutdown_connections 2024-04-26T01:54:58.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:54:58.344+0000 7fe18f000700 1 -- 172.21.15.37:0/1277517266 wait complete. 2024-04-26T01:54:58.345 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:54:58.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:58 smithi045 ceph-mon[32520]: pgmap v358: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.0 KiB/s wr, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:58.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:58 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:58.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:58 smithi071 ceph-mon[32578]: pgmap v358: 97 pgs: 97 active+clean; 629 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.0 KiB/s wr, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:54:58.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:58 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:54:59.220 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:54:59.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:54:59 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1277517266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:59.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:54:59 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1277517266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:54:59.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:54:59 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1277517266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:55:00.350 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:00.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:00 smithi037 ceph-mon[27906]: pgmap v359: 97 pgs: 97 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.7 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:55:00.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.499 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:00 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:00 smithi045 ceph-mon[32520]: pgmap v359: 97 pgs: 97 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.7 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:55:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:00 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:00 smithi071 ceph-mon[32578]: pgmap v359: 97 pgs: 97 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.7 KiB/s wr, 4 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T01:55:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:00 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:02.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:02 smithi037 ceph-mon[27906]: pgmap v360: 97 pgs: 97 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-04-26T01:55:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:02 smithi045 ceph-mon[32520]: pgmap v360: 97 pgs: 97 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-04-26T01:55:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:02 smithi071 ceph-mon[32578]: pgmap v360: 97 pgs: 97 active+clean; 634 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-04-26T01:55:02.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 -- 172.21.15.37:0/948047538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664101940 msgr2=0x7ff664101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:02.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 --2- 172.21.15.37:0/948047538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664101940 0x7ff664101dc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff650009990 tx=0x7ff650009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 -- 172.21.15.37:0/948047538 shutdown_connections 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 --2- 172.21.15.37:0/948047538 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff66410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 --2- 172.21.15.37:0/948047538 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664107990 0x7ff664107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 --2- 172.21.15.37:0/948047538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664101940 0x7ff664101dc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 -- 172.21.15.37:0/948047538 >> 172.21.15.37:0/948047538 conn(0x7ff6640788f0 msgr2=0x7ff664078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 -- 172.21.15.37:0/948047538 shutdown_connections 2024-04-26T01:55:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.852+0000 7ff668c00700 1 -- 172.21.15.37:0/948047538 wait complete. 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.853+0000 7ff668c00700 1 Processor -- start 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 -- start start 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664101940 0x7ff6641034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff664103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664107990 0x7ff664103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff664111b30 con 0x7ff664107990 2024-04-26T01:55:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641119b0 con 0x7ff664101940 2024-04-26T01:55:02.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.854+0000 7ff668c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff664111cb0 con 0x7ff664102300 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff662800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664107990 0x7ff664103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff664103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff664103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51102/0 (socket says 172.21.15.37:51102) 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 -- 172.21.15.37:0/3497545196 learned_addr learned my addr 172.21.15.37:0/3497545196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661e00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664101940 0x7ff6641034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 -- 172.21.15.37:0/3497545196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664101940 msgr2=0x7ff6641034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664101940 0x7ff6641034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 -- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664107990 msgr2=0x7ff664103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664107990 0x7ff664103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:02.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 -- 172.21.15.37:0/3497545196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6641a76b0 con 0x7ff664102300 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661e00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664101940 0x7ff6641034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.855+0000 7ff661400700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff664103a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6500057c0 tx=0x7ff6500057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.856+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65000fb00 con 0x7ff664102300 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.856+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65000fca0 con 0x7ff664102300 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.856+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6641a7940 con 0x7ff664102300 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.856+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff650017650 con 0x7ff664102300 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.856+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a7d90 con 0x7ff664102300 2024-04-26T01:55:02.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.858+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff664069730 con 0x7ff664102300 2024-04-26T01:55:02.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.859+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff650009d10 con 0x7ff664102300 2024-04-26T01:55:02.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.860+0000 7ff65aa00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff6480a3d50 0x7ff6480a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.861+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7ff6500d1470 con 0x7ff664102300 2024-04-26T01:55:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.861+0000 7ff661e00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff6480a3d50 0x7ff6480a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:02.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.861+0000 7ff661e00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff6480a3d50 0x7ff6480a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff654009680 tx=0x7ff654009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:02.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:02.865+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff650013070 con 0x7ff664102300 2024-04-26T01:55:03.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.105+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff66402df80 con 0x7ff664102300 2024-04-26T01:55:03.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.107+0000 7ff65aa00700 1 -- 172.21.15.37:0/3497545196 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22499 (secure 0 0 0) 0x7ff65009dd30 con 0x7ff664102300 2024-04-26T01:55:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:55:03.107 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:44.157091+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:55:03.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff6480a3d50 msgr2=0x7ff6480a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff6480a3d50 0x7ff6480a6210 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff654009680 tx=0x7ff654009260 comp rx=0 tx=0).stop 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 msgr2=0x7ff664103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff664103a30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6500057c0 tx=0x7ff6500057f0 comp rx=0 tx=0).stop 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 shutdown_connections 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff664107990 0x7ff664103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7ff6480a3d50 0x7ff6480a6210 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff664101940 0x7ff6641034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 --2- 172.21.15.37:0/3497545196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff664102300 0x7ff664103a30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.110+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 >> 172.21.15.37:0/3497545196 conn(0x7ff6640788f0 msgr2=0x7ff66406ab60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.111+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 shutdown_connections 2024-04-26T01:55:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:03.111+0000 7ff668c00700 1 -- 172.21.15.37:0/3497545196 wait complete. 2024-04-26T01:55:03.875 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:55:03.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:55:04.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:03 smithi071 ceph-mon[32578]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-04-26T01:55:04.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:04.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:03 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:04.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:03 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3497545196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:03 smithi037 ceph-mon[27906]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-04-26T01:55:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:04.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:03 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:03 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3497545196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:03 smithi045 ceph-mon[32520]: pgmap v361: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.4 KiB/s wr, 4 op/s 2024-04-26T01:55:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:03 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:03 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3497545196' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:04.370 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:05.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:05.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:05.030 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:04 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:05.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:05.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:05.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:04 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:55:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:55:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:04 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:55:06.122 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:05 smithi037 ceph-mon[27906]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T01:55:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:05 smithi045 ceph-mon[32520]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T01:55:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:05 smithi071 ceph-mon[32578]: pgmap v362: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T01:55:06.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 -- 172.21.15.37:0/230195037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610105870 msgr2=0x7f96101098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 --2- 172.21.15.37:0/230195037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610105870 0x7f96101098c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0099c0 tx=0x7f95fc009cd0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 -- 172.21.15.37:0/230195037 shutdown_connections 2024-04-26T01:55:06.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 --2- 172.21.15.37:0/230195037 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9610109fa0 0x7f9610111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 --2- 172.21.15.37:0/230195037 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9610104ec0 0x7f96101052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 --2- 172.21.15.37:0/230195037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610105870 0x7f96101098c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 -- 172.21.15.37:0/230195037 >> 172.21.15.37:0/230195037 conn(0x7f9610100750 msgr2=0x7f9610102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:06.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 -- 172.21.15.37:0/230195037 shutdown_connections 2024-04-26T01:55:06.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.647+0000 7f9617400700 1 -- 172.21.15.37:0/230195037 wait complete. 2024-04-26T01:55:06.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.648+0000 7f9617400700 1 Processor -- start 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 -- start start 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 0x7f961019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9610105870 0x7f961019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9610109fa0 0x7f961019fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9610114e60 con 0x7f9610104ec0 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9610114ce0 con 0x7f9610105870 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9617400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9610114fe0 con 0x7f9610109fa0 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.649+0000 7f9614a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 0x7f961019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:06.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 0x7f961019b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47104/0 (socket says 172.21.15.37:47104) 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 -- 172.21.15.37:0/2790944266 learned_addr learned my addr 172.21.15.37:0/2790944266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9615400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9610109fa0 0x7f961019fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f960fe00700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9610105870 0x7f961019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 -- 172.21.15.37:0/2790944266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9610109fa0 msgr2=0x7f961019fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9610109fa0 0x7f961019fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 -- 172.21.15.37:0/2790944266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9610105870 msgr2=0x7f961019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9610105870 0x7f961019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 -- 172.21.15.37:0/2790944266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96101a0330 con 0x7f9610104ec0 2024-04-26T01:55:06.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.650+0000 7f9614a00700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 0x7f961019b370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9a0 tx=0x7f960400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:06.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.651+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960400ca10 con 0x7f9610104ec0 2024-04-26T01:55:06.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.651+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f960400cbb0 con 0x7f9610104ec0 2024-04-26T01:55:06.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.651+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9604018680 con 0x7f9610104ec0 2024-04-26T01:55:06.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.651+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96101a0620 con 0x7f9610104ec0 2024-04-26T01:55:06.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.651+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96101a7ea0 con 0x7f9610104ec0 2024-04-26T01:55:06.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.652+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96101063a0 con 0x7f9610104ec0 2024-04-26T01:55:06.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.654+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9604010430 con 0x7f9610104ec0 2024-04-26T01:55:06.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.656+0000 7f960d600700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f80a3d50 0x7f95f80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:06.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.656+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f96040d06d0 con 0x7f9610104ec0 2024-04-26T01:55:06.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.656+0000 7f960fe00700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f80a3d50 0x7f95f80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:06.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.657+0000 7f960fe00700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f80a3d50 0x7f95f80a6210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:06.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.661+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f960409cf00 con 0x7f9610104ec0 2024-04-26T01:55:06.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.889+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9610062e80 con 0x7f95f80a3d50 2024-04-26T01:55:06.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.907+0000 7f960d600700 1 -- 172.21.15.37:0/2790944266 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258644 (secure 0 0 0) 0x7f9610062e80 con 0x7f95f80a3d50 2024-04-26T01:55:06.907 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:55:06.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.914+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f80a3d50 msgr2=0x7f95f80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.914+0000 7f9617400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f80a3d50 0x7f95f80a6210 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c00 tx=0x7f95fc005fb0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.914+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 msgr2=0x7f961019b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:06.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.914+0000 7f9617400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 0x7f961019b370 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f960400e9a0 tx=0x7f960400ecb0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 shutdown_connections 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9610104ec0 0x7f961019b370 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95f80a3d50 0x7f95f80a6210 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9610109fa0 0x7f961019fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 --2- 172.21.15.37:0/2790944266 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9610105870 0x7f961019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 >> 172.21.15.37:0/2790944266 conn(0x7f9610100750 msgr2=0x7f9610102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 shutdown_connections 2024-04-26T01:55:06.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:06.915+0000 7f9617400700 1 -- 172.21.15.37:0/2790944266 wait complete. 2024-04-26T01:55:06.916 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:55:07.726 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-04-26T01:55:06.539468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653735,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":217,"num_read_kb":223,"num_write":266,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362740,"kb_used_data":13560,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579084,"statfs":{"total":1151940427776,"available":1151568982016,"internally_reserved":0,"allocated":13885440,"data_stored":7756449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":22,"num_read_kb":21,"num_write":8,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005130"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177331+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:54:44.177331+0000","last_peered":"2024-04-26T01:54:44.177331+0000","last_clean":"2024-04-26T01:54:44.177331+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:54:44.177331+0000","last_undegraded":"2024-04-26T01:54:44.177331+0000","last_fullsized":"2024-04-26T01:54:44.177331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183121+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:54:44.183121+0000","last_peered":"2024-04-26T01:54:44.183121+0000","last_clean":"2024-04-26T01:54:44.183121+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:54:44.183121+0000","last_undegraded":"2024-04-26T01:54:44.183121+0000","last_fullsized":"2024-04-26T01:54:44.183121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183301+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:54:44.183301+0000","last_peered":"2024-04-26T01:54:44.183301+0000","last_clean":"2024-04-26T01:54:44.183301+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:54:44.183301+0000","last_undegraded":"2024-04-26T01:54:44.183301+0000","last_fullsized":"2024-04-26T01:54:44.183301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175139+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:54:44.175139+0000","last_peered":"2024-04-26T01:54:44.175139+0000","last_clean":"2024-04-26T01:54:44.175139+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:54:44.175139+0000","last_undegraded":"2024-04-26T01:54:44.175139+0000","last_fullsized":"2024-04-26T01:54:44.175139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178064+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:54:44.178064+0000","last_peered":"2024-04-26T01:54:44.178064+0000","last_clean":"2024-04-26T01:54:44.178064+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:54:44.178064+0000","last_undegraded":"2024-04-26T01:54:44.178064+0000","last_fullsized":"2024-04-26T01:54:44.178064+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376065+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:54:44.376065+0000","last_peered":"2024-04-26T01:54:44.376065+0000","last_clean":"2024-04-26T01:54:44.376065+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:54:44.376065+0000","last_undegraded":"2024-04-26T01:54:44.376065+0000","last_fullsized":"2024-04-26T01:54:44.376065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376691+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:54:44.376691+0000","last_peered":"2024-04-26T01:54:44.376691+0000","last_clean":"2024-04-26T01:54:44.376691+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:54:44.376691+0000","last_undegraded":"2024-04-26T01:54:44.376691+0000","last_fullsized":"2024-04-26T01:54:44.376691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.180976+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:54:44.180976+0000","last_peered":"2024-04-26T01:54:44.180976+0000","last_clean":"2024-04-26T01:54:44.180976+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:54:44.180976+0000","last_undegraded":"2024-04-26T01:54:44.180976+0000","last_fullsized":"2024-04-26T01:54:44.180976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177851+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:54:44.177851+0000","last_peered":"2024-04-26T01:54:44.177851+0000","last_clean":"2024-04-26T01:54:44.177851+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:54:44.177851+0000","last_undegraded":"2024-04-26T01:54:44.177851+0000","last_fullsized":"2024-04-26T01:54:44.177851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376790+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:54:44.376790+0000","last_peered":"2024-04-26T01:54:44.376790+0000","last_clean":"2024-04-26T01:54:44.376790+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:54:44.376790+0000","last_undegraded":"2024-04-26T01:54:44.376790+0000","last_fullsized":"2024-04-26T01:54:44.376790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377192+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:54:44.377192+0000","last_peered":"2024-04-26T01:54:44.377192+0000","last_clean":"2024-04-26T01:54:44.377192+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:54:44.377192+0000","last_undegraded":"2024-04-26T01:54:44.377192+0000","last_fullsized":"2024-04-26T01:54:44.377192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175007+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:54:44.175007+0000","last_peered":"2024-04-26T01:54:44.175007+0000","last_clean":"2024-04-26T01:54:44.175007+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:54:44.175007+0000","last_undegraded":"2024-04-26T01:54:44.175007+0000","last_fullsized":"2024-04-26T01:54:44.175007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377314+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:54:44.377314+0000","last_peered":"2024-04-26T01:54:44.377314+0000","last_clean":"2024-04-26T01:54:44.377314+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:54:44.377314+0000","last_undegraded":"2024-04-26T01:54:44.377314+0000","last_fullsized":"2024-04-26T01:54:44.377314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.185516+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:54:44.185516+0000","last_peered":"2024-04-26T01:54:44.185516+0000","last_clean":"2024-04-26T01:54:44.185516+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:54:44.185516+0000","last_undegraded":"2024-04-26T01:54:44.185516+0000","last_fullsized":"2024-04-26T01:54:44.185516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181158+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:54:44.181158+0000","last_peered":"2024-04-26T01:54:44.181158+0000","last_clean":"2024-04-26T01:54:44.181158+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:54:44.181158+0000","last_undegraded":"2024-04-26T01:54:44.181158+0000","last_fullsized":"2024-04-26T01:54:44.181158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177229+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:54:44.177229+0000","last_peered":"2024-04-26T01:54:44.177229+0000","last_clean":"2024-04-26T01:54:44.177229+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:54:44.177229+0000","last_undegraded":"2024-04-26T01:54:44.177229+0000","last_fullsized":"2024-04-26T01:54:44.177229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177542+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:54:44.177542+0000","last_peered":"2024-04-26T01:54:44.177542+0000","last_clean":"2024-04-26T01:54:44.177542+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:54:44.177542+0000","last_undegraded":"2024-04-26T01:54:44.177542+0000","last_fullsized":"2024-04-26T01:54:44.177542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.186797+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:54:44.186797+0000","last_peered":"2024-04-26T01:54:44.186797+0000","last_clean":"2024-04-26T01:54:44.186797+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:54:44.186797+0000","last_undegraded":"2024-04-26T01:54:44.186797+0000","last_fullsized":"2024-04-26T01:54:44.186797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175433+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:54:44.175433+0000","last_peered":"2024-04-26T01:54:44.175433+0000","last_clean":"2024-04-26T01:54:44.175433+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:54:44.175433+0000","last_undegraded":"2024-04-26T01:54:44.175433+0000","last_fullsized":"2024-04-26T01:54:44.175433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.187747+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:54:44.187747+0000","last_peered":"2024-04-26T01:54:44.187747+0000","last_clean":"2024-04-26T01:54:44.187747+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:54:44.187747+0000","last_undegraded":"2024-04-26T01:54:44.187747+0000","last_fullsized":"2024-04-26T01:54:44.187747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183768+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:54:44.183768+0000","last_peered":"2024-04-26T01:54:44.183768+0000","last_clean":"2024-04-26T01:54:44.183768+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:54:44.183768+0000","last_undegraded":"2024-04-26T01:54:44.183768+0000","last_fullsized":"2024-04-26T01:54:44.183768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.184509+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:54:44.184509+0000","last_peered":"2024-04-26T01:54:44.184509+0000","last_clean":"2024-04-26T01:54:44.184509+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:54:44.184509+0000","last_undegraded":"2024-04-26T01:54:44.184509+0000","last_fullsized":"2024-04-26T01:54:44.184509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.184277+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:54:44.184277+0000","last_peered":"2024-04-26T01:54:44.184277+0000","last_clean":"2024-04-26T01:54:44.184277+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:54:44.184277+0000","last_undegraded":"2024-04-26T01:54:44.184277+0000","last_fullsized":"2024-04-26T01:54:44.184277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178449+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:54:44.178449+0000","last_peered":"2024-04-26T01:54:44.178449+0000","last_clean":"2024-04-26T01:54:44.178449+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:54:44.178449+0000","last_undegraded":"2024-04-26T01:54:44.178449+0000","last_fullsized":"2024-04-26T01:54:44.178449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177350+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:54:44.177350+0000","last_peered":"2024-04-26T01:54:44.177350+0000","last_clean":"2024-04-26T01:54:44.177350+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:54:44.177350+0000","last_undegraded":"2024-04-26T01:54:44.177350+0000","last_fullsized":"2024-04-26T01:54:44.177350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178726+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:54:44.178726+0000","last_peered":"2024-04-26T01:54:44.178726+0000","last_clean":"2024-04-26T01:54:44.178726+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:54:44.178726+0000","last_undegraded":"2024-04-26T01:54:44.178726+0000","last_fullsized":"2024-04-26T01:54:44.178726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375167+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:54:44.375167+0000","last_peered":"2024-04-26T01:54:44.375167+0000","last_clean":"2024-04-26T01:54:44.375167+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:54:44.375167+0000","last_undegraded":"2024-04-26T01:54:44.375167+0000","last_fullsized":"2024-04-26T01:54:44.375167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374980+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:54:44.374980+0000","last_peered":"2024-04-26T01:54:44.374980+0000","last_clean":"2024-04-26T01:54:44.374980+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:54:44.374980+0000","last_undegraded":"2024-04-26T01:54:44.374980+0000","last_fullsized":"2024-04-26T01:54:44.374980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177639+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:54:44.177639+0000","last_peered":"2024-04-26T01:54:44.177639+0000","last_clean":"2024-04-26T01:54:44.177639+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:54:44.177639+0000","last_undegraded":"2024-04-26T01:54:44.177639+0000","last_fullsized":"2024-04-26T01:54:44.177639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375951+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:54:44.375951+0000","last_peered":"2024-04-26T01:54:44.375951+0000","last_clean":"2024-04-26T01:54:44.375951+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:54:44.375951+0000","last_undegraded":"2024-04-26T01:54:44.375951+0000","last_fullsized":"2024-04-26T01:54:44.375951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375054+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:54:44.375054+0000","last_peered":"2024-04-26T01:54:44.375054+0000","last_clean":"2024-04-26T01:54:44.375054+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:54:44.375054+0000","last_undegraded":"2024-04-26T01:54:44.375054+0000","last_fullsized":"2024-04-26T01:54:44.375054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376609+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:54:44.376609+0000","last_peered":"2024-04-26T01:54:44.376609+0000","last_clean":"2024-04-26T01:54:44.376609+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:54:44.376609+0000","last_undegraded":"2024-04-26T01:54:44.376609+0000","last_fullsized":"2024-04-26T01:54:44.376609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.180651+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:54:44.180651+0000","last_peered":"2024-04-26T01:54:44.180651+0000","last_clean":"2024-04-26T01:54:44.180651+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:54:44.180651+0000","last_undegraded":"2024-04-26T01:54:44.180651+0000","last_fullsized":"2024-04-26T01:54:44.180651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374905+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:54:44.374905+0000","last_peered":"2024-04-26T01:54:44.374905+0000","last_clean":"2024-04-26T01:54:44.374905+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:54:44.374905+0000","last_undegraded":"2024-04-26T01:54:44.374905+0000","last_fullsized":"2024-04-26T01:54:44.374905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183067+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:54:44.183067+0000","last_peered":"2024-04-26T01:54:44.183067+0000","last_clean":"2024-04-26T01:54:44.183067+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:54:44.183067+0000","last_undegraded":"2024-04-26T01:54:44.183067+0000","last_fullsized":"2024-04-26T01:54:44.183067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376782+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:54:44.376782+0000","last_peered":"2024-04-26T01:54:44.376782+0000","last_clean":"2024-04-26T01:54:44.376782+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:54:44.376782+0000","last_undegraded":"2024-04-26T01:54:44.376782+0000","last_fullsized":"2024-04-26T01:54:44.376782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177698+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:54:44.177698+0000","last_peered":"2024-04-26T01:54:44.177698+0000","last_clean":"2024-04-26T01:54:44.177698+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:54:44.177698+0000","last_undegraded":"2024-04-26T01:54:44.177698+0000","last_fullsized":"2024-04-26T01:54:44.177698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377200+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:54:44.377200+0000","last_peered":"2024-04-26T01:54:44.377200+0000","last_clean":"2024-04-26T01:54:44.377200+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:54:44.377200+0000","last_undegraded":"2024-04-26T01:54:44.377200+0000","last_fullsized":"2024-04-26T01:54:44.377200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.185825+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:54:44.185825+0000","last_peered":"2024-04-26T01:54:44.185825+0000","last_clean":"2024-04-26T01:54:44.185825+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:54:44.185825+0000","last_undegraded":"2024-04-26T01:54:44.185825+0000","last_fullsized":"2024-04-26T01:54:44.185825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375158+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:54:44.375158+0000","last_peered":"2024-04-26T01:54:44.375158+0000","last_clean":"2024-04-26T01:54:44.375158+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:54:44.375158+0000","last_undegraded":"2024-04-26T01:54:44.375158+0000","last_fullsized":"2024-04-26T01:54:44.375158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177511+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:54:44.177511+0000","last_peered":"2024-04-26T01:54:44.177511+0000","last_clean":"2024-04-26T01:54:44.177511+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:54:44.177511+0000","last_undegraded":"2024-04-26T01:54:44.177511+0000","last_fullsized":"2024-04-26T01:54:44.177511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'28","reported_seq":129,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:55:01.588639+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:55:01.588639+0000","last_peered":"2024-04-26T01:55:01.588639+0000","last_clean":"2024-04-26T01:55:01.588639+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:55:01.588639+0000","last_undegraded":"2024-04-26T01:55:01.588639+0000","last_fullsized":"2024-04-26T01:55:01.588639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62335,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":75,"num_read_kb":98,"num_write":28,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178020+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:54:44.178020+0000","last_peered":"2024-04-26T01:54:44.178020+0000","last_clean":"2024-04-26T01:54:44.178020+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:54:44.178020+0000","last_undegraded":"2024-04-26T01:54:44.178020+0000","last_fullsized":"2024-04-26T01:54:44.178020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175166+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:54:44.175166+0000","last_peered":"2024-04-26T01:54:44.175166+0000","last_clean":"2024-04-26T01:54:44.175166+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:54:44.175166+0000","last_undegraded":"2024-04-26T01:54:44.175166+0000","last_fullsized":"2024-04-26T01:54:44.175166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376383+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:54:44.376383+0000","last_peered":"2024-04-26T01:54:44.376383+0000","last_clean":"2024-04-26T01:54:44.376383+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:54:44.376383+0000","last_undegraded":"2024-04-26T01:54:44.376383+0000","last_fullsized":"2024-04-26T01:54:44.376383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374916+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:54:44.374916+0000","last_peered":"2024-04-26T01:54:44.374916+0000","last_clean":"2024-04-26T01:54:44.374916+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:54:44.374916+0000","last_undegraded":"2024-04-26T01:54:44.374916+0000","last_fullsized":"2024-04-26T01:54:44.374916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377044+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:54:44.377044+0000","last_peered":"2024-04-26T01:54:44.377044+0000","last_clean":"2024-04-26T01:54:44.377044+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:54:44.377044+0000","last_undegraded":"2024-04-26T01:54:44.377044+0000","last_fullsized":"2024-04-26T01:54:44.377044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174926+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:54:44.174926+0000","last_peered":"2024-04-26T01:54:44.174926+0000","last_clean":"2024-04-26T01:54:44.174926+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:54:44.174926+0000","last_undegraded":"2024-04-26T01:54:44.174926+0000","last_fullsized":"2024-04-26T01:54:44.174926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376333+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:54:44.376333+0000","last_peered":"2024-04-26T01:54:44.376333+0000","last_clean":"2024-04-26T01:54:44.376333+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:54:44.376333+0000","last_undegraded":"2024-04-26T01:54:44.376333+0000","last_fullsized":"2024-04-26T01:54:44.376333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.197929+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:54:44.197929+0000","last_peered":"2024-04-26T01:54:44.197929+0000","last_clean":"2024-04-26T01:54:44.197929+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:54:44.197929+0000","last_undegraded":"2024-04-26T01:54:44.197929+0000","last_fullsized":"2024-04-26T01:54:44.197929+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374726+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:54:44.374726+0000","last_peered":"2024-04-26T01:54:44.374726+0000","last_clean":"2024-04-26T01:54:44.374726+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:54:44.374726+0000","last_undegraded":"2024-04-26T01:54:44.374726+0000","last_fullsized":"2024-04-26T01:54:44.374726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178251+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:54:44.178251+0000","last_peered":"2024-04-26T01:54:44.178251+0000","last_clean":"2024-04-26T01:54:44.178251+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:54:44.178251+0000","last_undegraded":"2024-04-26T01:54:44.178251+0000","last_fullsized":"2024-04-26T01:54:44.178251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177065+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:54:44.177065+0000","last_peered":"2024-04-26T01:54:44.177065+0000","last_clean":"2024-04-26T01:54:44.177065+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:54:44.177065+0000","last_undegraded":"2024-04-26T01:54:44.177065+0000","last_fullsized":"2024-04-26T01:54:44.177065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.182809+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:54:44.182809+0000","last_peered":"2024-04-26T01:54:44.182809+0000","last_clean":"2024-04-26T01:54:44.182809+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:54:44.182809+0000","last_undegraded":"2024-04-26T01:54:44.182809+0000","last_fullsized":"2024-04-26T01:54:44.182809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376282+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:54:44.376282+0000","last_peered":"2024-04-26T01:54:44.376282+0000","last_clean":"2024-04-26T01:54:44.376282+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:54:44.376282+0000","last_undegraded":"2024-04-26T01:54:44.376282+0000","last_fullsized":"2024-04-26T01:54:44.376282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376500+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:54:44.376500+0000","last_peered":"2024-04-26T01:54:44.376500+0000","last_clean":"2024-04-26T01:54:44.376500+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:54:44.376500+0000","last_undegraded":"2024-04-26T01:54:44.376500+0000","last_fullsized":"2024-04-26T01:54:44.376500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376134+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:54:44.376134+0000","last_peered":"2024-04-26T01:54:44.376134+0000","last_clean":"2024-04-26T01:54:44.376134+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:54:44.376134+0000","last_undegraded":"2024-04-26T01:54:44.376134+0000","last_fullsized":"2024-04-26T01:54:44.376134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375297+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:54:44.375297+0000","last_peered":"2024-04-26T01:54:44.375297+0000","last_clean":"2024-04-26T01:54:44.375297+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:54:44.375297+0000","last_undegraded":"2024-04-26T01:54:44.375297+0000","last_fullsized":"2024-04-26T01:54:44.375297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376220+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:54:44.376220+0000","last_peered":"2024-04-26T01:54:44.376220+0000","last_clean":"2024-04-26T01:54:44.376220+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:54:44.376220+0000","last_undegraded":"2024-04-26T01:54:44.376220+0000","last_fullsized":"2024-04-26T01:54:44.376220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183358+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:54:44.183358+0000","last_peered":"2024-04-26T01:54:44.183358+0000","last_clean":"2024-04-26T01:54:44.183358+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:54:44.183358+0000","last_undegraded":"2024-04-26T01:54:44.183358+0000","last_fullsized":"2024-04-26T01:54:44.183358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175028+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:54:44.175028+0000","last_peered":"2024-04-26T01:54:44.175028+0000","last_clean":"2024-04-26T01:54:44.175028+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:54:44.175028+0000","last_undegraded":"2024-04-26T01:54:44.175028+0000","last_fullsized":"2024-04-26T01:54:44.175028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376592+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:54:44.376592+0000","last_peered":"2024-04-26T01:54:44.376592+0000","last_clean":"2024-04-26T01:54:44.376592+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:54:44.376592+0000","last_undegraded":"2024-04-26T01:54:44.376592+0000","last_fullsized":"2024-04-26T01:54:44.376592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376240+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:54:44.376240+0000","last_peered":"2024-04-26T01:54:44.376240+0000","last_clean":"2024-04-26T01:54:44.376240+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:54:44.376240+0000","last_undegraded":"2024-04-26T01:54:44.376240+0000","last_fullsized":"2024-04-26T01:54:44.376240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376706+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:54:44.376706+0000","last_peered":"2024-04-26T01:54:44.376706+0000","last_clean":"2024-04-26T01:54:44.376706+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:54:44.376706+0000","last_undegraded":"2024-04-26T01:54:44.376706+0000","last_fullsized":"2024-04-26T01:54:44.376706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178281+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:54:44.178281+0000","last_peered":"2024-04-26T01:54:44.178281+0000","last_clean":"2024-04-26T01:54:44.178281+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:54:44.178281+0000","last_undegraded":"2024-04-26T01:54:44.178281+0000","last_fullsized":"2024-04-26T01:54:44.178281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377018+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:54:44.377018+0000","last_peered":"2024-04-26T01:54:44.377018+0000","last_clean":"2024-04-26T01:54:44.377018+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:54:44.377018+0000","last_undegraded":"2024-04-26T01:54:44.377018+0000","last_fullsized":"2024-04-26T01:54:44.377018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:56.034137+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:54:56.034137+0000","last_peered":"2024-04-26T01:54:56.034137+0000","last_clean":"2024-04-26T01:54:56.034137+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:54:56.034137+0000","last_undegraded":"2024-04-26T01:54:56.034137+0000","last_fullsized":"2024-04-26T01:54:56.034137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181904+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:54:44.181904+0000","last_peered":"2024-04-26T01:54:44.181904+0000","last_clean":"2024-04-26T01:54:44.181904+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:54:44.181904+0000","last_undegraded":"2024-04-26T01:54:44.181904+0000","last_fullsized":"2024-04-26T01:54:44.181904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175181+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:54:44.175181+0000","last_peered":"2024-04-26T01:54:44.175181+0000","last_clean":"2024-04-26T01:54:44.175181+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:54:44.175181+0000","last_undegraded":"2024-04-26T01:54:44.175181+0000","last_fullsized":"2024-04-26T01:54:44.175181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.379735+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:54:44.379735+0000","last_peered":"2024-04-26T01:54:44.379735+0000","last_clean":"2024-04-26T01:54:44.379735+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:54:44.379735+0000","last_undegraded":"2024-04-26T01:54:44.379735+0000","last_fullsized":"2024-04-26T01:54:44.379735+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.189980+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:54:44.189980+0000","last_peered":"2024-04-26T01:54:44.189980+0000","last_clean":"2024-04-26T01:54:44.189980+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:54:44.189980+0000","last_undegraded":"2024-04-26T01:54:44.189980+0000","last_fullsized":"2024-04-26T01:54:44.189980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177082+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:54:44.177082+0000","last_peered":"2024-04-26T01:54:44.177082+0000","last_clean":"2024-04-26T01:54:44.177082+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:54:44.177082+0000","last_undegraded":"2024-04-26T01:54:44.177082+0000","last_fullsized":"2024-04-26T01:54:44.177082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181016+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:54:44.181016+0000","last_peered":"2024-04-26T01:54:44.181016+0000","last_clean":"2024-04-26T01:54:44.181016+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:54:44.181016+0000","last_undegraded":"2024-04-26T01:54:44.181016+0000","last_fullsized":"2024-04-26T01:54:44.181016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.176912+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:54:44.176912+0000","last_peered":"2024-04-26T01:54:44.176912+0000","last_clean":"2024-04-26T01:54:44.176912+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:54:44.176912+0000","last_undegraded":"2024-04-26T01:54:44.176912+0000","last_fullsized":"2024-04-26T01:54:44.176912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174987+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:54:44.174987+0000","last_peered":"2024-04-26T01:54:44.174987+0000","last_clean":"2024-04-26T01:54:44.174987+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:54:44.174987+0000","last_undegraded":"2024-04-26T01:54:44.174987+0000","last_fullsized":"2024-04-26T01:54:44.174987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.382932+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:54:44.382932+0000","last_peered":"2024-04-26T01:54:44.382932+0000","last_clean":"2024-04-26T01:54:44.382932+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:54:44.382932+0000","last_undegraded":"2024-04-26T01:54:44.382932+0000","last_fullsized":"2024-04-26T01:54:44.382932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177943+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:54:44.177943+0000","last_peered":"2024-04-26T01:54:44.177943+0000","last_clean":"2024-04-26T01:54:44.177943+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:54:44.177943+0000","last_undegraded":"2024-04-26T01:54:44.177943+0000","last_fullsized":"2024-04-26T01:54:44.177943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376891+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:54:44.376891+0000","last_peered":"2024-04-26T01:54:44.376891+0000","last_clean":"2024-04-26T01:54:44.376891+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:54:44.376891+0000","last_undegraded":"2024-04-26T01:54:44.376891+0000","last_fullsized":"2024-04-26T01:54:44.376891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175227+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:54:44.175227+0000","last_peered":"2024-04-26T01:54:44.175227+0000","last_clean":"2024-04-26T01:54:44.175227+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:54:44.175227+0000","last_undegraded":"2024-04-26T01:54:44.175227+0000","last_fullsized":"2024-04-26T01:54:44.175227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377331+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:54:44.377331+0000","last_peered":"2024-04-26T01:54:44.377331+0000","last_clean":"2024-04-26T01:54:44.377331+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:54:44.377331+0000","last_undegraded":"2024-04-26T01:54:44.377331+0000","last_fullsized":"2024-04-26T01:54:44.377331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374975+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:54:44.374975+0000","last_peered":"2024-04-26T01:54:44.374975+0000","last_clean":"2024-04-26T01:54:44.374975+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:54:44.374975+0000","last_undegraded":"2024-04-26T01:54:44.374975+0000","last_fullsized":"2024-04-26T01:54:44.374975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377093+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:54:44.377093+0000","last_peered":"2024-04-26T01:54:44.377093+0000","last_clean":"2024-04-26T01:54:44.377093+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:54:44.377093+0000","last_undegraded":"2024-04-26T01:54:44.377093+0000","last_fullsized":"2024-04-26T01:54:44.377093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377037+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:54:44.377037+0000","last_peered":"2024-04-26T01:54:44.377037+0000","last_clean":"2024-04-26T01:54:44.377037+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:54:44.377037+0000","last_undegraded":"2024-04-26T01:54:44.377037+0000","last_fullsized":"2024-04-26T01:54:44.377037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377292+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:54:44.377292+0000","last_peered":"2024-04-26T01:54:44.377292+0000","last_clean":"2024-04-26T01:54:44.377292+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:54:44.377292+0000","last_undegraded":"2024-04-26T01:54:44.377292+0000","last_fullsized":"2024-04-26T01:54:44.377292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.189043+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:54:44.189043+0000","last_peered":"2024-04-26T01:54:44.189043+0000","last_clean":"2024-04-26T01:54:44.189043+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:54:44.189043+0000","last_undegraded":"2024-04-26T01:54:44.189043+0000","last_fullsized":"2024-04-26T01:54:44.189043+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174916+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:54:44.174916+0000","last_peered":"2024-04-26T01:54:44.174916+0000","last_clean":"2024-04-26T01:54:44.174916+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:54:44.174916+0000","last_undegraded":"2024-04-26T01:54:44.174916+0000","last_fullsized":"2024-04-26T01:54:44.174916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174911+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:54:44.174911+0000","last_peered":"2024-04-26T01:54:44.174911+0000","last_clean":"2024-04-26T01:54:44.174911+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:54:44.174911+0000","last_undegraded":"2024-04-26T01:54:44.174911+0000","last_fullsized":"2024-04-26T01:54:44.174911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.182401+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:54:44.182401+0000","last_peered":"2024-04-26T01:54:44.182401+0000","last_clean":"2024-04-26T01:54:44.182401+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:54:44.182401+0000","last_undegraded":"2024-04-26T01:54:44.182401+0000","last_fullsized":"2024-04-26T01:54:44.182401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377173+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:54:44.377173+0000","last_peered":"2024-04-26T01:54:44.377173+0000","last_clean":"2024-04-26T01:54:44.377173+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:54:44.377173+0000","last_undegraded":"2024-04-26T01:54:44.377173+0000","last_fullsized":"2024-04-26T01:54:44.377173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177005+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:54:44.177005+0000","last_peered":"2024-04-26T01:54:44.177005+0000","last_clean":"2024-04-26T01:54:44.177005+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:54:44.177005+0000","last_undegraded":"2024-04-26T01:54:44.177005+0000","last_fullsized":"2024-04-26T01:54:44.177005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175542+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:54:44.175542+0000","last_peered":"2024-04-26T01:54:44.175542+0000","last_clean":"2024-04-26T01:54:44.175542+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:54:44.175542+0000","last_undegraded":"2024-04-26T01:54:44.175542+0000","last_fullsized":"2024-04-26T01:54:44.175542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.190885+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:54:44.190885+0000","last_peered":"2024-04-26T01:54:44.190885+0000","last_clean":"2024-04-26T01:54:44.190885+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:54:44.190885+0000","last_undegraded":"2024-04-26T01:54:44.190885+0000","last_fullsized":"2024-04-26T01:54:44.190885+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375114+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:54:44.375114+0000","last_peered":"2024-04-26T01:54:44.375114+0000","last_clean":"2024-04-26T01:54:44.375114+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:54:44.375114+0000","last_undegraded":"2024-04-26T01:54:44.375114+0000","last_fullsized":"2024-04-26T01:54:44.375114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.180691+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:54:44.180691+0000","last_peered":"2024-04-26T01:54:44.180691+0000","last_clean":"2024-04-26T01:54:44.180691+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:54:44.180691+0000","last_undegraded":"2024-04-26T01:54:44.180691+0000","last_fullsized":"2024-04-26T01:54:44.180691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177416+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:54:44.177416+0000","last_peered":"2024-04-26T01:54:44.177416+0000","last_clean":"2024-04-26T01:54:44.177416+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:54:44.177416+0000","last_undegraded":"2024-04-26T01:54:44.177416+0000","last_fullsized":"2024-04-26T01:54:44.177416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377314+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:54:44.377314+0000","last_peered":"2024-04-26T01:54:44.377314+0000","last_clean":"2024-04-26T01:54:44.377314+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:54:44.377314+0000","last_undegraded":"2024-04-26T01:54:44.377314+0000","last_fullsized":"2024-04-26T01:54:44.377314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.188650+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:54:44.188650+0000","last_peered":"2024-04-26T01:54:44.188650+0000","last_clean":"2024-04-26T01:54:44.188650+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:54:44.188650+0000","last_undegraded":"2024-04-26T01:54:44.188650+0000","last_fullsized":"2024-04-26T01:54:44.188650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63133,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":81,"num_read_kb":104,"num_write":83,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":189399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351007,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":482937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1135951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":483519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":483541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299443,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1074004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":483054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":545272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":483027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":545306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":483275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888132,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":482959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724608,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1073604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:55:07.731 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:55:07.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:07 smithi037 ceph-mon[27906]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-26T01:55:07.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:07 smithi037 ceph-mon[27906]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:55:08.237 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:07 smithi045 ceph-mon[32520]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-26T01:55:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:07 smithi045 ceph-mon[32520]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:55:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:07 smithi071 ceph-mon[32578]: pgmap v363: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-04-26T01:55:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:07 smithi071 ceph-mon[32578]: from='client.14526 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:55:10.126 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:09 smithi037 ceph-mon[27906]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T01:55:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:09 smithi045 ceph-mon[32520]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T01:55:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:09 smithi071 ceph-mon[32578]: pgmap v364: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.687+0000 7fe6f9800700 1 -- 172.21.15.37:0/3002734576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4109fa0 msgr2=0x7fe6f4111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.687+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3002734576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4109fa0 0x7fe6f4111b70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe6ec00b0d0 tx=0x7fe6ec00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 -- 172.21.15.37:0/3002734576 shutdown_connections 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3002734576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4109fa0 0x7fe6f4111b70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3002734576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f41052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3002734576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f4105870 0x7fe6f41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 -- 172.21.15.37:0/3002734576 >> 172.21.15.37:0/3002734576 conn(0x7fe6f4100750 msgr2=0x7fe6f4102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 -- 172.21.15.37:0/3002734576 shutdown_connections 2024-04-26T01:55:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.688+0000 7fe6f9800700 1 -- 172.21.15.37:0/3002734576 wait complete. 2024-04-26T01:55:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.689+0000 7fe6f9800700 1 Processor -- start 2024-04-26T01:55:10.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.689+0000 7fe6f9800700 1 -- start start 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f9800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f9800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 0x7fe6f419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f9800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f4109fa0 0x7fe6f419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f9800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4114e30 con 0x7fe6f4105870 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f9800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4114cb0 con 0x7fe6f4104ec0 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f9800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe6f4114fb0 con 0x7fe6f4109fa0 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 0x7fe6f419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:10.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54816/0 (socket says 172.21.15.37:54816) 2024-04-26T01:55:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f2000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 0x7fe6f419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47666/0 (socket says 172.21.15.37:47666) 2024-04-26T01:55:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f2a00700 1 -- 172.21.15.37:0/3293615561 learned_addr learned my addr 172.21.15.37:0/3293615561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:10.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.690+0000 7fe6f3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f4109fa0 0x7fe6f419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2a00700 1 -- 172.21.15.37:0/3293615561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f4109fa0 msgr2=0x7fe6f419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2a00700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f4109fa0 0x7fe6f419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2a00700 1 -- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 msgr2=0x7fe6f419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2a00700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 0x7fe6f419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2a00700 1 -- 172.21.15.37:0/3293615561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6f41a02a0 con 0x7fe6f4104ec0 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2000700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 0x7fe6f419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.691+0000 7fe6f2a00700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f419b2e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400eb30 tx=0x7fe6e400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:10.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.692+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e400cbd0 con 0x7fe6f4104ec0 2024-04-26T01:55:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.692+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6f41a0590 con 0x7fe6f4104ec0 2024-04-26T01:55:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.692+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6e40044d0 con 0x7fe6f4104ec0 2024-04-26T01:55:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.692+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6f41a7e10 con 0x7fe6f4104ec0 2024-04-26T01:55:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.692+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e4003e30 con 0x7fe6f4104ec0 2024-04-26T01:55:10.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.693+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6f4069730 con 0x7fe6f4104ec0 2024-04-26T01:55:10.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.695+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe6e4004000 con 0x7fe6f4104ec0 2024-04-26T01:55:10.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.697+0000 7fe6eb400700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe6d80a3cd0 0x7fe6d80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:10.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.697+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe6e40d0930 con 0x7fe6f4104ec0 2024-04-26T01:55:10.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.697+0000 7fe6f2000700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe6d80a3cd0 0x7fe6d80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:10.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.698+0000 7fe6f2000700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe6d80a3cd0 0x7fe6d80a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4103ed0 tx=0x7fe6e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:10.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.701+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6e40042f0 con 0x7fe6f4104ec0 2024-04-26T01:55:10.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.934+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe6f4062e80 con 0x7fe6d80a3cd0 2024-04-26T01:55:10.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.950+0000 7fe6eb400700 1 -- 172.21.15.37:0/3293615561 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258644 (secure 0 0 0) 0x7fe6f4062e80 con 0x7fe6d80a3cd0 2024-04-26T01:55:10.951 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.957+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe6d80a3cd0 msgr2=0x7fe6d80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.957+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe6d80a3cd0 0x7fe6d80a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4103ed0 tx=0x7fe6e0009260 comp rx=0 tx=0).stop 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.957+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 msgr2=0x7fe6f419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.957+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f419b2e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe6e400eb30 tx=0x7fe6e400ee40 comp rx=0 tx=0).stop 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 shutdown_connections 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe6d80a3cd0 0x7fe6d80a6190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe6f4109fa0 0x7fe6f419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe6f4104ec0 0x7fe6f419b2e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 --2- 172.21.15.37:0/3293615561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6f4105870 0x7fe6f419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 >> 172.21.15.37:0/3293615561 conn(0x7fe6f4100750 msgr2=0x7fe6f4101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 shutdown_connections 2024-04-26T01:55:10.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:10.958+0000 7fe6f9800700 1 -- 172.21.15.37:0/3293615561 wait complete. 2024-04-26T01:55:10.959 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:55:11.993 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-26T01:55:10.541446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653735,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":227,"num_read_kb":228,"num_write":266,"num_write_kb":3027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":362740,"kb_used_data":13560,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124579084,"statfs":{"total":1151940427776,"available":1151568982016,"internally_reserved":0,"allocated":13885440,"data_stored":7756449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":357414202},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4257,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":3,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005057"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177331+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:54:44.177331+0000","last_peered":"2024-04-26T01:54:44.177331+0000","last_clean":"2024-04-26T01:54:44.177331+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:54:44.177331+0000","last_undegraded":"2024-04-26T01:54:44.177331+0000","last_fullsized":"2024-04-26T01:54:44.177331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183121+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:54:44.183121+0000","last_peered":"2024-04-26T01:54:44.183121+0000","last_clean":"2024-04-26T01:54:44.183121+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:54:44.183121+0000","last_undegraded":"2024-04-26T01:54:44.183121+0000","last_fullsized":"2024-04-26T01:54:44.183121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183301+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:54:44.183301+0000","last_peered":"2024-04-26T01:54:44.183301+0000","last_clean":"2024-04-26T01:54:44.183301+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:54:44.183301+0000","last_undegraded":"2024-04-26T01:54:44.183301+0000","last_fullsized":"2024-04-26T01:54:44.183301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175139+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:54:44.175139+0000","last_peered":"2024-04-26T01:54:44.175139+0000","last_clean":"2024-04-26T01:54:44.175139+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:54:44.175139+0000","last_undegraded":"2024-04-26T01:54:44.175139+0000","last_fullsized":"2024-04-26T01:54:44.175139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178064+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:54:44.178064+0000","last_peered":"2024-04-26T01:54:44.178064+0000","last_clean":"2024-04-26T01:54:44.178064+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:54:44.178064+0000","last_undegraded":"2024-04-26T01:54:44.178064+0000","last_fullsized":"2024-04-26T01:54:44.178064+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376065+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:54:44.376065+0000","last_peered":"2024-04-26T01:54:44.376065+0000","last_clean":"2024-04-26T01:54:44.376065+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:54:44.376065+0000","last_undegraded":"2024-04-26T01:54:44.376065+0000","last_fullsized":"2024-04-26T01:54:44.376065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376691+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:54:44.376691+0000","last_peered":"2024-04-26T01:54:44.376691+0000","last_clean":"2024-04-26T01:54:44.376691+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:54:44.376691+0000","last_undegraded":"2024-04-26T01:54:44.376691+0000","last_fullsized":"2024-04-26T01:54:44.376691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.180976+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:54:44.180976+0000","last_peered":"2024-04-26T01:54:44.180976+0000","last_clean":"2024-04-26T01:54:44.180976+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:54:44.180976+0000","last_undegraded":"2024-04-26T01:54:44.180976+0000","last_fullsized":"2024-04-26T01:54:44.180976+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177851+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:54:44.177851+0000","last_peered":"2024-04-26T01:54:44.177851+0000","last_clean":"2024-04-26T01:54:44.177851+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:54:44.177851+0000","last_undegraded":"2024-04-26T01:54:44.177851+0000","last_fullsized":"2024-04-26T01:54:44.177851+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376790+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:54:44.376790+0000","last_peered":"2024-04-26T01:54:44.376790+0000","last_clean":"2024-04-26T01:54:44.376790+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:54:44.376790+0000","last_undegraded":"2024-04-26T01:54:44.376790+0000","last_fullsized":"2024-04-26T01:54:44.376790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377192+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:54:44.377192+0000","last_peered":"2024-04-26T01:54:44.377192+0000","last_clean":"2024-04-26T01:54:44.377192+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:54:44.377192+0000","last_undegraded":"2024-04-26T01:54:44.377192+0000","last_fullsized":"2024-04-26T01:54:44.377192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175007+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:54:44.175007+0000","last_peered":"2024-04-26T01:54:44.175007+0000","last_clean":"2024-04-26T01:54:44.175007+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:54:44.175007+0000","last_undegraded":"2024-04-26T01:54:44.175007+0000","last_fullsized":"2024-04-26T01:54:44.175007+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377314+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:54:44.377314+0000","last_peered":"2024-04-26T01:54:44.377314+0000","last_clean":"2024-04-26T01:54:44.377314+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:54:44.377314+0000","last_undegraded":"2024-04-26T01:54:44.377314+0000","last_fullsized":"2024-04-26T01:54:44.377314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.185516+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:54:44.185516+0000","last_peered":"2024-04-26T01:54:44.185516+0000","last_clean":"2024-04-26T01:54:44.185516+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:54:44.185516+0000","last_undegraded":"2024-04-26T01:54:44.185516+0000","last_fullsized":"2024-04-26T01:54:44.185516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181158+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:54:44.181158+0000","last_peered":"2024-04-26T01:54:44.181158+0000","last_clean":"2024-04-26T01:54:44.181158+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:54:44.181158+0000","last_undegraded":"2024-04-26T01:54:44.181158+0000","last_fullsized":"2024-04-26T01:54:44.181158+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177229+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:54:44.177229+0000","last_peered":"2024-04-26T01:54:44.177229+0000","last_clean":"2024-04-26T01:54:44.177229+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:54:44.177229+0000","last_undegraded":"2024-04-26T01:54:44.177229+0000","last_fullsized":"2024-04-26T01:54:44.177229+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177542+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:54:44.177542+0000","last_peered":"2024-04-26T01:54:44.177542+0000","last_clean":"2024-04-26T01:54:44.177542+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:54:44.177542+0000","last_undegraded":"2024-04-26T01:54:44.177542+0000","last_fullsized":"2024-04-26T01:54:44.177542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.186797+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:54:44.186797+0000","last_peered":"2024-04-26T01:54:44.186797+0000","last_clean":"2024-04-26T01:54:44.186797+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:54:44.186797+0000","last_undegraded":"2024-04-26T01:54:44.186797+0000","last_fullsized":"2024-04-26T01:54:44.186797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175433+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:54:44.175433+0000","last_peered":"2024-04-26T01:54:44.175433+0000","last_clean":"2024-04-26T01:54:44.175433+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:54:44.175433+0000","last_undegraded":"2024-04-26T01:54:44.175433+0000","last_fullsized":"2024-04-26T01:54:44.175433+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.187747+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:54:44.187747+0000","last_peered":"2024-04-26T01:54:44.187747+0000","last_clean":"2024-04-26T01:54:44.187747+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:54:44.187747+0000","last_undegraded":"2024-04-26T01:54:44.187747+0000","last_fullsized":"2024-04-26T01:54:44.187747+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183768+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:54:44.183768+0000","last_peered":"2024-04-26T01:54:44.183768+0000","last_clean":"2024-04-26T01:54:44.183768+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:54:44.183768+0000","last_undegraded":"2024-04-26T01:54:44.183768+0000","last_fullsized":"2024-04-26T01:54:44.183768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.184509+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:54:44.184509+0000","last_peered":"2024-04-26T01:54:44.184509+0000","last_clean":"2024-04-26T01:54:44.184509+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:54:44.184509+0000","last_undegraded":"2024-04-26T01:54:44.184509+0000","last_fullsized":"2024-04-26T01:54:44.184509+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.184277+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:54:44.184277+0000","last_peered":"2024-04-26T01:54:44.184277+0000","last_clean":"2024-04-26T01:54:44.184277+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:54:44.184277+0000","last_undegraded":"2024-04-26T01:54:44.184277+0000","last_fullsized":"2024-04-26T01:54:44.184277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178449+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:54:44.178449+0000","last_peered":"2024-04-26T01:54:44.178449+0000","last_clean":"2024-04-26T01:54:44.178449+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:54:44.178449+0000","last_undegraded":"2024-04-26T01:54:44.178449+0000","last_fullsized":"2024-04-26T01:54:44.178449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177350+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:54:44.177350+0000","last_peered":"2024-04-26T01:54:44.177350+0000","last_clean":"2024-04-26T01:54:44.177350+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:54:44.177350+0000","last_undegraded":"2024-04-26T01:54:44.177350+0000","last_fullsized":"2024-04-26T01:54:44.177350+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178726+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:54:44.178726+0000","last_peered":"2024-04-26T01:54:44.178726+0000","last_clean":"2024-04-26T01:54:44.178726+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:54:44.178726+0000","last_undegraded":"2024-04-26T01:54:44.178726+0000","last_fullsized":"2024-04-26T01:54:44.178726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375167+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:54:44.375167+0000","last_peered":"2024-04-26T01:54:44.375167+0000","last_clean":"2024-04-26T01:54:44.375167+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:54:44.375167+0000","last_undegraded":"2024-04-26T01:54:44.375167+0000","last_fullsized":"2024-04-26T01:54:44.375167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374980+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:54:44.374980+0000","last_peered":"2024-04-26T01:54:44.374980+0000","last_clean":"2024-04-26T01:54:44.374980+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:54:44.374980+0000","last_undegraded":"2024-04-26T01:54:44.374980+0000","last_fullsized":"2024-04-26T01:54:44.374980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177639+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:54:44.177639+0000","last_peered":"2024-04-26T01:54:44.177639+0000","last_clean":"2024-04-26T01:54:44.177639+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:54:44.177639+0000","last_undegraded":"2024-04-26T01:54:44.177639+0000","last_fullsized":"2024-04-26T01:54:44.177639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375951+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:54:44.375951+0000","last_peered":"2024-04-26T01:54:44.375951+0000","last_clean":"2024-04-26T01:54:44.375951+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:54:44.375951+0000","last_undegraded":"2024-04-26T01:54:44.375951+0000","last_fullsized":"2024-04-26T01:54:44.375951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375054+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:54:44.375054+0000","last_peered":"2024-04-26T01:54:44.375054+0000","last_clean":"2024-04-26T01:54:44.375054+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:54:44.375054+0000","last_undegraded":"2024-04-26T01:54:44.375054+0000","last_fullsized":"2024-04-26T01:54:44.375054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376609+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:54:44.376609+0000","last_peered":"2024-04-26T01:54:44.376609+0000","last_clean":"2024-04-26T01:54:44.376609+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:54:44.376609+0000","last_undegraded":"2024-04-26T01:54:44.376609+0000","last_fullsized":"2024-04-26T01:54:44.376609+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.180651+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:54:44.180651+0000","last_peered":"2024-04-26T01:54:44.180651+0000","last_clean":"2024-04-26T01:54:44.180651+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:54:44.180651+0000","last_undegraded":"2024-04-26T01:54:44.180651+0000","last_fullsized":"2024-04-26T01:54:44.180651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374905+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:54:44.374905+0000","last_peered":"2024-04-26T01:54:44.374905+0000","last_clean":"2024-04-26T01:54:44.374905+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:54:44.374905+0000","last_undegraded":"2024-04-26T01:54:44.374905+0000","last_fullsized":"2024-04-26T01:54:44.374905+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183067+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:54:44.183067+0000","last_peered":"2024-04-26T01:54:44.183067+0000","last_clean":"2024-04-26T01:54:44.183067+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:54:44.183067+0000","last_undegraded":"2024-04-26T01:54:44.183067+0000","last_fullsized":"2024-04-26T01:54:44.183067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376782+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:54:44.376782+0000","last_peered":"2024-04-26T01:54:44.376782+0000","last_clean":"2024-04-26T01:54:44.376782+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:54:44.376782+0000","last_undegraded":"2024-04-26T01:54:44.376782+0000","last_fullsized":"2024-04-26T01:54:44.376782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177698+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:54:44.177698+0000","last_peered":"2024-04-26T01:54:44.177698+0000","last_clean":"2024-04-26T01:54:44.177698+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:54:44.177698+0000","last_undegraded":"2024-04-26T01:54:44.177698+0000","last_fullsized":"2024-04-26T01:54:44.177698+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377200+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:54:44.377200+0000","last_peered":"2024-04-26T01:54:44.377200+0000","last_clean":"2024-04-26T01:54:44.377200+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:54:44.377200+0000","last_undegraded":"2024-04-26T01:54:44.377200+0000","last_fullsized":"2024-04-26T01:54:44.377200+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.185825+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:54:44.185825+0000","last_peered":"2024-04-26T01:54:44.185825+0000","last_clean":"2024-04-26T01:54:44.185825+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:54:44.185825+0000","last_undegraded":"2024-04-26T01:54:44.185825+0000","last_fullsized":"2024-04-26T01:54:44.185825+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375158+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:54:44.375158+0000","last_peered":"2024-04-26T01:54:44.375158+0000","last_clean":"2024-04-26T01:54:44.375158+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:54:44.375158+0000","last_undegraded":"2024-04-26T01:54:44.375158+0000","last_fullsized":"2024-04-26T01:54:44.375158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177511+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:54:44.177511+0000","last_peered":"2024-04-26T01:54:44.177511+0000","last_clean":"2024-04-26T01:54:44.177511+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:54:44.177511+0000","last_undegraded":"2024-04-26T01:54:44.177511+0000","last_fullsized":"2024-04-26T01:54:44.177511+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'28","reported_seq":139,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:55:06.596083+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:55:06.596083+0000","last_peered":"2024-04-26T01:55:06.596083+0000","last_clean":"2024-04-26T01:55:06.596083+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:55:06.596083+0000","last_undegraded":"2024-04-26T01:55:06.596083+0000","last_fullsized":"2024-04-26T01:55:06.596083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62335,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":85,"num_read_kb":103,"num_write":28,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178020+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:54:44.178020+0000","last_peered":"2024-04-26T01:54:44.178020+0000","last_clean":"2024-04-26T01:54:44.178020+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:54:44.178020+0000","last_undegraded":"2024-04-26T01:54:44.178020+0000","last_fullsized":"2024-04-26T01:54:44.178020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175166+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:54:44.175166+0000","last_peered":"2024-04-26T01:54:44.175166+0000","last_clean":"2024-04-26T01:54:44.175166+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:54:44.175166+0000","last_undegraded":"2024-04-26T01:54:44.175166+0000","last_fullsized":"2024-04-26T01:54:44.175166+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376383+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:54:44.376383+0000","last_peered":"2024-04-26T01:54:44.376383+0000","last_clean":"2024-04-26T01:54:44.376383+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:54:44.376383+0000","last_undegraded":"2024-04-26T01:54:44.376383+0000","last_fullsized":"2024-04-26T01:54:44.376383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374916+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:54:44.374916+0000","last_peered":"2024-04-26T01:54:44.374916+0000","last_clean":"2024-04-26T01:54:44.374916+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:54:44.374916+0000","last_undegraded":"2024-04-26T01:54:44.374916+0000","last_fullsized":"2024-04-26T01:54:44.374916+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377044+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:54:44.377044+0000","last_peered":"2024-04-26T01:54:44.377044+0000","last_clean":"2024-04-26T01:54:44.377044+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:54:44.377044+0000","last_undegraded":"2024-04-26T01:54:44.377044+0000","last_fullsized":"2024-04-26T01:54:44.377044+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174926+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:54:44.174926+0000","last_peered":"2024-04-26T01:54:44.174926+0000","last_clean":"2024-04-26T01:54:44.174926+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:54:44.174926+0000","last_undegraded":"2024-04-26T01:54:44.174926+0000","last_fullsized":"2024-04-26T01:54:44.174926+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376333+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:54:44.376333+0000","last_peered":"2024-04-26T01:54:44.376333+0000","last_clean":"2024-04-26T01:54:44.376333+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:54:44.376333+0000","last_undegraded":"2024-04-26T01:54:44.376333+0000","last_fullsized":"2024-04-26T01:54:44.376333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.197929+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:54:44.197929+0000","last_peered":"2024-04-26T01:54:44.197929+0000","last_clean":"2024-04-26T01:54:44.197929+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:54:44.197929+0000","last_undegraded":"2024-04-26T01:54:44.197929+0000","last_fullsized":"2024-04-26T01:54:44.197929+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374726+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:54:44.374726+0000","last_peered":"2024-04-26T01:54:44.374726+0000","last_clean":"2024-04-26T01:54:44.374726+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:54:44.374726+0000","last_undegraded":"2024-04-26T01:54:44.374726+0000","last_fullsized":"2024-04-26T01:54:44.374726+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178251+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:54:44.178251+0000","last_peered":"2024-04-26T01:54:44.178251+0000","last_clean":"2024-04-26T01:54:44.178251+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:54:44.178251+0000","last_undegraded":"2024-04-26T01:54:44.178251+0000","last_fullsized":"2024-04-26T01:54:44.178251+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177065+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:54:44.177065+0000","last_peered":"2024-04-26T01:54:44.177065+0000","last_clean":"2024-04-26T01:54:44.177065+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:54:44.177065+0000","last_undegraded":"2024-04-26T01:54:44.177065+0000","last_fullsized":"2024-04-26T01:54:44.177065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.182809+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:54:44.182809+0000","last_peered":"2024-04-26T01:54:44.182809+0000","last_clean":"2024-04-26T01:54:44.182809+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:54:44.182809+0000","last_undegraded":"2024-04-26T01:54:44.182809+0000","last_fullsized":"2024-04-26T01:54:44.182809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376282+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:54:44.376282+0000","last_peered":"2024-04-26T01:54:44.376282+0000","last_clean":"2024-04-26T01:54:44.376282+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:54:44.376282+0000","last_undegraded":"2024-04-26T01:54:44.376282+0000","last_fullsized":"2024-04-26T01:54:44.376282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376500+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:54:44.376500+0000","last_peered":"2024-04-26T01:54:44.376500+0000","last_clean":"2024-04-26T01:54:44.376500+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:54:44.376500+0000","last_undegraded":"2024-04-26T01:54:44.376500+0000","last_fullsized":"2024-04-26T01:54:44.376500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376134+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:54:44.376134+0000","last_peered":"2024-04-26T01:54:44.376134+0000","last_clean":"2024-04-26T01:54:44.376134+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:54:44.376134+0000","last_undegraded":"2024-04-26T01:54:44.376134+0000","last_fullsized":"2024-04-26T01:54:44.376134+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375297+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:54:44.375297+0000","last_peered":"2024-04-26T01:54:44.375297+0000","last_clean":"2024-04-26T01:54:44.375297+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:54:44.375297+0000","last_undegraded":"2024-04-26T01:54:44.375297+0000","last_fullsized":"2024-04-26T01:54:44.375297+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376220+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:54:44.376220+0000","last_peered":"2024-04-26T01:54:44.376220+0000","last_clean":"2024-04-26T01:54:44.376220+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:54:44.376220+0000","last_undegraded":"2024-04-26T01:54:44.376220+0000","last_fullsized":"2024-04-26T01:54:44.376220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.183358+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:54:44.183358+0000","last_peered":"2024-04-26T01:54:44.183358+0000","last_clean":"2024-04-26T01:54:44.183358+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:54:44.183358+0000","last_undegraded":"2024-04-26T01:54:44.183358+0000","last_fullsized":"2024-04-26T01:54:44.183358+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175028+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:54:44.175028+0000","last_peered":"2024-04-26T01:54:44.175028+0000","last_clean":"2024-04-26T01:54:44.175028+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:54:44.175028+0000","last_undegraded":"2024-04-26T01:54:44.175028+0000","last_fullsized":"2024-04-26T01:54:44.175028+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376592+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:54:44.376592+0000","last_peered":"2024-04-26T01:54:44.376592+0000","last_clean":"2024-04-26T01:54:44.376592+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:54:44.376592+0000","last_undegraded":"2024-04-26T01:54:44.376592+0000","last_fullsized":"2024-04-26T01:54:44.376592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376240+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:54:44.376240+0000","last_peered":"2024-04-26T01:54:44.376240+0000","last_clean":"2024-04-26T01:54:44.376240+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:54:44.376240+0000","last_undegraded":"2024-04-26T01:54:44.376240+0000","last_fullsized":"2024-04-26T01:54:44.376240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376706+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:54:44.376706+0000","last_peered":"2024-04-26T01:54:44.376706+0000","last_clean":"2024-04-26T01:54:44.376706+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:54:44.376706+0000","last_undegraded":"2024-04-26T01:54:44.376706+0000","last_fullsized":"2024-04-26T01:54:44.376706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178281+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:54:44.178281+0000","last_peered":"2024-04-26T01:54:44.178281+0000","last_clean":"2024-04-26T01:54:44.178281+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:54:44.178281+0000","last_undegraded":"2024-04-26T01:54:44.178281+0000","last_fullsized":"2024-04-26T01:54:44.178281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377018+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:54:44.377018+0000","last_peered":"2024-04-26T01:54:44.377018+0000","last_clean":"2024-04-26T01:54:44.377018+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:54:44.377018+0000","last_undegraded":"2024-04-26T01:54:44.377018+0000","last_fullsized":"2024-04-26T01:54:44.377018+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'9","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:56.034137+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:54:56.034137+0000","last_peered":"2024-04-26T01:54:56.034137+0000","last_clean":"2024-04-26T01:54:56.034137+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:54:56.034137+0000","last_undegraded":"2024-04-26T01:54:56.034137+0000","last_fullsized":"2024-04-26T01:54:56.034137+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181904+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:54:44.181904+0000","last_peered":"2024-04-26T01:54:44.181904+0000","last_clean":"2024-04-26T01:54:44.181904+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:54:44.181904+0000","last_undegraded":"2024-04-26T01:54:44.181904+0000","last_fullsized":"2024-04-26T01:54:44.181904+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175181+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:54:44.175181+0000","last_peered":"2024-04-26T01:54:44.175181+0000","last_clean":"2024-04-26T01:54:44.175181+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:54:44.175181+0000","last_undegraded":"2024-04-26T01:54:44.175181+0000","last_fullsized":"2024-04-26T01:54:44.175181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.379735+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:54:44.379735+0000","last_peered":"2024-04-26T01:54:44.379735+0000","last_clean":"2024-04-26T01:54:44.379735+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:54:44.379735+0000","last_undegraded":"2024-04-26T01:54:44.379735+0000","last_fullsized":"2024-04-26T01:54:44.379735+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.189980+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:54:44.189980+0000","last_peered":"2024-04-26T01:54:44.189980+0000","last_clean":"2024-04-26T01:54:44.189980+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:54:44.189980+0000","last_undegraded":"2024-04-26T01:54:44.189980+0000","last_fullsized":"2024-04-26T01:54:44.189980+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177082+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:54:44.177082+0000","last_peered":"2024-04-26T01:54:44.177082+0000","last_clean":"2024-04-26T01:54:44.177082+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:54:44.177082+0000","last_undegraded":"2024-04-26T01:54:44.177082+0000","last_fullsized":"2024-04-26T01:54:44.177082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181016+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:54:44.181016+0000","last_peered":"2024-04-26T01:54:44.181016+0000","last_clean":"2024-04-26T01:54:44.181016+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:54:44.181016+0000","last_undegraded":"2024-04-26T01:54:44.181016+0000","last_fullsized":"2024-04-26T01:54:44.181016+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.176912+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:54:44.176912+0000","last_peered":"2024-04-26T01:54:44.176912+0000","last_clean":"2024-04-26T01:54:44.176912+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:54:44.176912+0000","last_undegraded":"2024-04-26T01:54:44.176912+0000","last_fullsized":"2024-04-26T01:54:44.176912+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174987+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:54:44.174987+0000","last_peered":"2024-04-26T01:54:44.174987+0000","last_clean":"2024-04-26T01:54:44.174987+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:54:44.174987+0000","last_undegraded":"2024-04-26T01:54:44.174987+0000","last_fullsized":"2024-04-26T01:54:44.174987+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.382932+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:54:44.382932+0000","last_peered":"2024-04-26T01:54:44.382932+0000","last_clean":"2024-04-26T01:54:44.382932+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:54:44.382932+0000","last_undegraded":"2024-04-26T01:54:44.382932+0000","last_fullsized":"2024-04-26T01:54:44.382932+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177943+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:54:44.177943+0000","last_peered":"2024-04-26T01:54:44.177943+0000","last_clean":"2024-04-26T01:54:44.177943+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:54:44.177943+0000","last_undegraded":"2024-04-26T01:54:44.177943+0000","last_fullsized":"2024-04-26T01:54:44.177943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376891+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:54:44.376891+0000","last_peered":"2024-04-26T01:54:44.376891+0000","last_clean":"2024-04-26T01:54:44.376891+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:54:44.376891+0000","last_undegraded":"2024-04-26T01:54:44.376891+0000","last_fullsized":"2024-04-26T01:54:44.376891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175227+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:54:44.175227+0000","last_peered":"2024-04-26T01:54:44.175227+0000","last_clean":"2024-04-26T01:54:44.175227+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:54:44.175227+0000","last_undegraded":"2024-04-26T01:54:44.175227+0000","last_fullsized":"2024-04-26T01:54:44.175227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377331+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:54:44.377331+0000","last_peered":"2024-04-26T01:54:44.377331+0000","last_clean":"2024-04-26T01:54:44.377331+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:54:44.377331+0000","last_undegraded":"2024-04-26T01:54:44.377331+0000","last_fullsized":"2024-04-26T01:54:44.377331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374975+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:54:44.374975+0000","last_peered":"2024-04-26T01:54:44.374975+0000","last_clean":"2024-04-26T01:54:44.374975+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:54:44.374975+0000","last_undegraded":"2024-04-26T01:54:44.374975+0000","last_fullsized":"2024-04-26T01:54:44.374975+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377093+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:54:44.377093+0000","last_peered":"2024-04-26T01:54:44.377093+0000","last_clean":"2024-04-26T01:54:44.377093+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:54:44.377093+0000","last_undegraded":"2024-04-26T01:54:44.377093+0000","last_fullsized":"2024-04-26T01:54:44.377093+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377037+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:54:44.377037+0000","last_peered":"2024-04-26T01:54:44.377037+0000","last_clean":"2024-04-26T01:54:44.377037+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:54:44.377037+0000","last_undegraded":"2024-04-26T01:54:44.377037+0000","last_fullsized":"2024-04-26T01:54:44.377037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377292+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:54:44.377292+0000","last_peered":"2024-04-26T01:54:44.377292+0000","last_clean":"2024-04-26T01:54:44.377292+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:54:44.377292+0000","last_undegraded":"2024-04-26T01:54:44.377292+0000","last_fullsized":"2024-04-26T01:54:44.377292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.189043+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:54:44.189043+0000","last_peered":"2024-04-26T01:54:44.189043+0000","last_clean":"2024-04-26T01:54:44.189043+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:54:44.189043+0000","last_undegraded":"2024-04-26T01:54:44.189043+0000","last_fullsized":"2024-04-26T01:54:44.189043+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174916+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:54:44.174916+0000","last_peered":"2024-04-26T01:54:44.174916+0000","last_clean":"2024-04-26T01:54:44.174916+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:54:44.174916+0000","last_undegraded":"2024-04-26T01:54:44.174916+0000","last_fullsized":"2024-04-26T01:54:44.174916+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174911+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:54:44.174911+0000","last_peered":"2024-04-26T01:54:44.174911+0000","last_clean":"2024-04-26T01:54:44.174911+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:54:44.174911+0000","last_undegraded":"2024-04-26T01:54:44.174911+0000","last_fullsized":"2024-04-26T01:54:44.174911+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.182401+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:54:44.182401+0000","last_peered":"2024-04-26T01:54:44.182401+0000","last_clean":"2024-04-26T01:54:44.182401+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:54:44.182401+0000","last_undegraded":"2024-04-26T01:54:44.182401+0000","last_fullsized":"2024-04-26T01:54:44.182401+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377173+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:54:44.377173+0000","last_peered":"2024-04-26T01:54:44.377173+0000","last_clean":"2024-04-26T01:54:44.377173+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:54:44.377173+0000","last_undegraded":"2024-04-26T01:54:44.377173+0000","last_fullsized":"2024-04-26T01:54:44.377173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177005+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:54:44.177005+0000","last_peered":"2024-04-26T01:54:44.177005+0000","last_clean":"2024-04-26T01:54:44.177005+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:54:44.177005+0000","last_undegraded":"2024-04-26T01:54:44.177005+0000","last_fullsized":"2024-04-26T01:54:44.177005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175542+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:54:44.175542+0000","last_peered":"2024-04-26T01:54:44.175542+0000","last_clean":"2024-04-26T01:54:44.175542+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:54:44.175542+0000","last_undegraded":"2024-04-26T01:54:44.175542+0000","last_fullsized":"2024-04-26T01:54:44.175542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.190885+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:54:44.190885+0000","last_peered":"2024-04-26T01:54:44.190885+0000","last_clean":"2024-04-26T01:54:44.190885+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:54:44.190885+0000","last_undegraded":"2024-04-26T01:54:44.190885+0000","last_fullsized":"2024-04-26T01:54:44.190885+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375114+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:54:44.375114+0000","last_peered":"2024-04-26T01:54:44.375114+0000","last_clean":"2024-04-26T01:54:44.375114+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:54:44.375114+0000","last_undegraded":"2024-04-26T01:54:44.375114+0000","last_fullsized":"2024-04-26T01:54:44.375114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.180691+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:54:44.180691+0000","last_peered":"2024-04-26T01:54:44.180691+0000","last_clean":"2024-04-26T01:54:44.180691+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:54:44.180691+0000","last_undegraded":"2024-04-26T01:54:44.180691+0000","last_fullsized":"2024-04-26T01:54:44.180691+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.177416+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:54:44.177416+0000","last_peered":"2024-04-26T01:54:44.177416+0000","last_clean":"2024-04-26T01:54:44.177416+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:54:44.177416+0000","last_undegraded":"2024-04-26T01:54:44.177416+0000","last_fullsized":"2024-04-26T01:54:44.177416+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377314+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:54:44.377314+0000","last_peered":"2024-04-26T01:54:44.377314+0000","last_clean":"2024-04-26T01:54:44.377314+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:54:44.377314+0000","last_undegraded":"2024-04-26T01:54:44.377314+0000","last_fullsized":"2024-04-26T01:54:44.377314+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.188650+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:54:44.188650+0000","last_peered":"2024-04-26T01:54:44.188650+0000","last_clean":"2024-04-26T01:54:44.188650+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:54:44.188650+0000","last_undegraded":"2024-04-26T01:54:44.188650+0000","last_fullsized":"2024-04-26T01:54:44.188650+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63133,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":91,"num_read_kb":109,"num_write":83,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":189399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351008,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":482937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612643,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1135951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":483519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037806,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":483541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1074004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462970,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":483054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574934,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":545272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738459,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":483027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":545306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":483275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888133,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":482959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1073604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:55:11.998 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:55:11.999 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T01:55:11.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T01:55:12.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:11 smithi037 ceph-mon[27906]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-26T01:55:12.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:11 smithi037 ceph-mon[27906]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:55:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:11 smithi045 ceph-mon[32520]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-26T01:55:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:11 smithi045 ceph-mon[32520]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:55:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:11 smithi071 ceph-mon[32578]: pgmap v365: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-04-26T01:55:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:11 smithi071 ceph-mon[32578]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:55:12.506 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:14.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:13 smithi037 ceph-mon[27906]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:55:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:13 smithi045 ceph-mon[32520]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:55:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:13 smithi071 ceph-mon[32578]: pgmap v366: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:55:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 -- 172.21.15.37:0/743806213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 msgr2=0x7efd80102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 --2- 172.21.15.37:0/743806213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80102ee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd740099c0 tx=0x7efd74009cd0 comp rx=0 tx=0).stop 2024-04-26T01:55:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 -- 172.21.15.37:0/743806213 shutdown_connections 2024-04-26T01:55:14.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 --2- 172.21.15.37:0/743806213 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd80104660 0x7efd8010af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 --2- 172.21.15.37:0/743806213 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80102ee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 --2- 172.21.15.37:0/743806213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.858+0000 7efd87e00700 1 -- 172.21.15.37:0/743806213 >> 172.21.15.37:0/743806213 conn(0x7efd800fe060 msgr2=0x7efd801004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.859+0000 7efd87e00700 1 -- 172.21.15.37:0/743806213 shutdown_connections 2024-04-26T01:55:14.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.859+0000 7efd87e00700 1 -- 172.21.15.37:0/743806213 wait complete. 2024-04-26T01:55:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.860+0000 7efd87e00700 1 Processor -- start 2024-04-26T01:55:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.860+0000 7efd87e00700 1 -- start start 2024-04-26T01:55:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd87e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd87e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd87e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd80104660 0x7efd801a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:14.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd87e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd8010e530 con 0x7efd80103cc0 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd87e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efd8010e3b0 con 0x7efd80102ac0 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd87e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efd8010e6b0 con 0x7efd80104660 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd84a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd85400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd85400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54842/0 (socket says 172.21.15.37:54842) 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd85400700 1 -- 172.21.15.37:0/3154497121 learned_addr learned my addr 172.21.15.37:0/3154497121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd84a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80199060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47684/0 (socket says 172.21.15.37:47684) 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.861+0000 7efd85e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd80104660 0x7efd801a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd84a00700 1 -- 172.21.15.37:0/3154497121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd80104660 msgr2=0x7efd801a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd84a00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd80104660 0x7efd801a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd84a00700 1 -- 172.21.15.37:0/3154497121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 msgr2=0x7efd80198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd84a00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:14.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd84a00700 1 -- 172.21.15.37:0/3154497121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd801a0780 con 0x7efd80103cc0 2024-04-26T01:55:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd85400700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd84a00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80199060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00ea20 tx=0x7efd6c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:14.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c00cbd0 con 0x7efd80103cc0 2024-04-26T01:55:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd6c00ee60 con 0x7efd80103cc0 2024-04-26T01:55:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd6c018680 con 0x7efd80103cc0 2024-04-26T01:55:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.862+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd801a0a70 con 0x7efd80103cc0 2024-04-26T01:55:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.863+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd801a0f60 con 0x7efd80103cc0 2024-04-26T01:55:14.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.864+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd80050fc0 con 0x7efd80103cc0 2024-04-26T01:55:14.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.866+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7efd6c010430 con 0x7efd80103cc0 2024-04-26T01:55:14.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.867+0000 7efd7e000700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efd680a3e00 0x7efd680a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:14.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.868+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7efd6c0d0830 con 0x7efd80103cc0 2024-04-26T01:55:14.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.868+0000 7efd85400700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efd680a3e00 0x7efd680a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:14.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.869+0000 7efd85400700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efd680a3e00 0x7efd680a62c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd74000c00 tx=0x7efd74005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:14.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:14.871+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd6c09d060 con 0x7efd80103cc0 2024-04-26T01:55:15.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.189+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7efd8002e1d0 con 0x7efd80103cc0 2024-04-26T01:55:15.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.193+0000 7efd7e000700 1 -- 172.21.15.37:0/3154497121 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7efd6c09c960 con 0x7efd80103cc0 2024-04-26T01:55:15.193 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-04-26T01:55:15.193 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQCDCStmVtNvCxAAGrkr22RNEY0naPeSrIu2Hw== 2024-04-26T01:55:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.196+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efd680a3e00 msgr2=0x7efd680a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:15.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.196+0000 7efd87e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efd680a3e00 0x7efd680a62c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efd74000c00 tx=0x7efd74005fb0 comp rx=0 tx=0).stop 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.196+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 msgr2=0x7efd80199060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.196+0000 7efd87e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80199060 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00ea20 tx=0x7efd6c00ed30 comp rx=0 tx=0).stop 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 shutdown_connections 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efd680a3e00 0x7efd680a62c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efd80104660 0x7efd801a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efd80102ac0 0x7efd80198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:15.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 --2- 172.21.15.37:0/3154497121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd80103cc0 0x7efd80199060 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:15.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 >> 172.21.15.37:0/3154497121 conn(0x7efd800fe060 msgr2=0x7efd80106fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:15.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 shutdown_connections 2024-04-26T01:55:15.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:15.197+0000 7efd87e00700 1 -- 172.21.15.37:0/3154497121 wait complete. 2024-04-26T01:55:16.085 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:55:16.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T01:55:16.085 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T01:55:16.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:15 smithi037 ceph-mon[27906]: pgmap v367: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:55:16.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:15 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3154497121' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:55:16.195 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:15 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3154497121' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:55:16.213 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T01:55:16.213 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T01:55:16.213 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T01:55:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:15 smithi045 ceph-mon[32520]: pgmap v367: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:55:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:15 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3154497121' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:55:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:15 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3154497121' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:55:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:15 smithi071 ceph-mon[32578]: pgmap v367: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:55:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:15 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3154497121' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T01:55:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:15 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3154497121' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T01:55:16.703 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:17 smithi037 ceph-mon[27906]: pgmap v368: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:17 smithi045 ceph-mon[32520]: pgmap v368: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:17 smithi071 ceph-mon[32578]: pgmap v368: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 -- 172.21.15.37:0/1322079545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 msgr2=0x7f1e20113390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 --2- 172.21.15.37:0/1322079545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e20113390 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c009990 tx=0x7f1e0c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 -- 172.21.15.37:0/1322079545 shutdown_connections 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 --2- 172.21.15.37:0/1322079545 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e20076f70 0x7f1e20077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 --2- 172.21.15.37:0/1322079545 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201138d0 0x7f1e20115cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 --2- 172.21.15.37:0/1322079545 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e20113390 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 -- 172.21.15.37:0/1322079545 >> 172.21.15.37:0/1322079545 conn(0x7f1e20100720 msgr2=0x7f1e20102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 -- 172.21.15.37:0/1322079545 shutdown_connections 2024-04-26T01:55:19.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.005+0000 7f1e26c00700 1 -- 172.21.15.37:0/1322079545 wait complete. 2024-04-26T01:55:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.006+0000 7f1e26c00700 1 Processor -- start 2024-04-26T01:55:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.006+0000 7f1e26c00700 1 -- start start 2024-04-26T01:55:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e26c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e20076f70 0x7f1e2019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e26c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e2019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e26c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201138d0 0x7f1e201a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e26c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20119260 con 0x7f1e20076f70 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e26c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201190e0 con 0x7f1e20077890 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e26c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1e201193e0 con 0x7f1e201138d0 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e2019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e1fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e20076f70 0x7f1e2019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e1f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e2019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54868/0 (socket says 172.21.15.37:54868) 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e1f400700 1 -- 172.21.15.37:0/3945526825 learned_addr learned my addr 172.21.15.37:0/3945526825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.007+0000 7f1e24c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201138d0 0x7f1e201a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1f400700 1 -- 172.21.15.37:0/3945526825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201138d0 msgr2=0x7f1e201a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1f400700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201138d0 0x7f1e201a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1f400700 1 -- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e20076f70 msgr2=0x7f1e2019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1f400700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e20076f70 0x7f1e2019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1f400700 1 -- 172.21.15.37:0/3945526825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e201a4760 con 0x7f1e20077890 2024-04-26T01:55:19.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1f400700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e2019fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0057f0 tx=0x7f1e0c00bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c026070 con 0x7f1e20077890 2024-04-26T01:55:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e0c021410 con 0x7f1e20077890 2024-04-26T01:55:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c017650 con 0x7f1e20077890 2024-04-26T01:55:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201a49f0 con 0x7f1e20077890 2024-04-26T01:55:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.008+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201ac2d0 con 0x7f1e20077890 2024-04-26T01:55:19.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.009+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e20069730 con 0x7f1e20077890 2024-04-26T01:55:19.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.011+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1e0c029070 con 0x7f1e20077890 2024-04-26T01:55:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.013+0000 7f1e1cc00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1e040a3cd0 0x7f1e040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.013+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f1e0c0d1260 con 0x7f1e20077890 2024-04-26T01:55:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.013+0000 7f1e1fe00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1e040a3cd0 0x7f1e040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:19.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.014+0000 7f1e1fe00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1e040a3cd0 0x7f1e040a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009680 tx=0x7f1e10009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:19.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.015+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e0c013070 con 0x7f1e20077890 2024-04-26T01:55:19.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.295+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f1e2002df80 con 0x7f1e20077890 2024-04-26T01:55:19.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.301+0000 7f1e1cc00700 1 -- 172.21.15.37:0/3945526825 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264991 (secure 0 0 0) 0x7f1e0c09dbb0 con 0x7f1e20077890 2024-04-26T01:55:19.305 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:55:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.306+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1e040a3cd0 msgr2=0x7f1e040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.306+0000 7f1e26c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1e040a3cd0 0x7f1e040a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009680 tx=0x7f1e10009260 comp rx=0 tx=0).stop 2024-04-26T01:55:19.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.306+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 msgr2=0x7f1e2019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e2019fc90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1e0c0057f0 tx=0x7f1e0c00bfd0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 shutdown_connections 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e20076f70 0x7f1e2019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1e040a3cd0 0x7f1e040a6190 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1e201138d0 0x7f1e201a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 --2- 172.21.15.37:0/3945526825 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1e20077890 0x7f1e2019fc90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 >> 172.21.15.37:0/3945526825 conn(0x7f1e20100720 msgr2=0x7f1e20102b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 shutdown_connections 2024-04-26T01:55:19.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:19.307+0000 7f1e26c00700 1 -- 172.21.15.37:0/3945526825 wait complete. 2024-04-26T01:55:20.061 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":691602839},{"type":"v1","addr":"172.21.15.37:6801","nonce":691602839}]},"active_addr":"172.21.15.37:6801/691602839","active_change":"2024-04-26T01:45:42.363509+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24112,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":457714458}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":29145428}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":793487795}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":4056686696}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3015884750}]}]} 2024-04-26T01:55:20.067 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T01:55:20.067 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T01:55:20.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:55:20.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:19 smithi037 ceph-mon[27906]: pgmap v369: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:20.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:19 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3945526825' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:55:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:19 smithi045 ceph-mon[32520]: pgmap v369: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:19 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3945526825' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:55:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:19 smithi071 ceph-mon[32578]: pgmap v369: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:19 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3945526825' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T01:55:20.537 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:21 smithi037 ceph-mon[27906]: pgmap v370: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:21 smithi045 ceph-mon[32520]: pgmap v370: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:21 smithi071 ceph-mon[32578]: pgmap v370: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.082+0000 7f7fa8200700 1 -- 172.21.15.37:0/1286398795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0108a30 msgr2=0x7f7fa0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.082+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1286398795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0108a30 0x7f7fa0108e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7f94009990 tx=0x7f7f94009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 -- 172.21.15.37:0/1286398795 shutdown_connections 2024-04-26T01:55:23.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1286398795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0108a30 0x7f7fa0108e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1286398795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa006b0f0 0x7f7fa010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1286398795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa006a750 0x7f7fa006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 -- 172.21.15.37:0/1286398795 >> 172.21.15.37:0/1286398795 conn(0x7f7fa00788f0 msgr2=0x7f7fa0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 -- 172.21.15.37:0/1286398795 shutdown_connections 2024-04-26T01:55:23.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.083+0000 7f7fa8200700 1 -- 172.21.15.37:0/1286398795 wait complete. 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 Processor -- start 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 -- start start 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa006a750 0x7f7fa0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 0x7f7fa0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa0108a30 0x7f7fa0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0112bd0 con 0x7f7fa006b0f0 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0112a50 con 0x7f7fa006a750 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.084+0000 7f7fa8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0112d50 con 0x7f7fa0108a30 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 0x7f7fa0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:23.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 0x7f7fa0101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39050/0 (socket says 172.21.15.37:39050) 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 -- 172.21.15.37:0/1407930997 learned_addr learned my addr 172.21.15.37:0/1407930997 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa5800700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa006a750 0x7f7fa0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa6200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa0108a30 0x7f7fa0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 -- 172.21.15.37:0/1407930997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa0108a30 msgr2=0x7f7fa0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa0108a30 0x7f7fa0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 -- 172.21.15.37:0/1407930997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa006a750 msgr2=0x7f7fa0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa006a750 0x7f7fa0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa4e00700 1 -- 172.21.15.37:0/1407930997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a75e0 con 0x7f7fa006b0f0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.085+0000 7f7fa6200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa0108a30 0x7f7fa0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.086+0000 7f7fa4e00700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 0x7f7fa0101f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00e9a0 tx=0x7f7f8c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.086+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c00cbd0 con 0x7f7fa006b0f0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.086+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f8c00cd70 con 0x7f7fa006b0f0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.086+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a78d0 con 0x7f7fa006b0f0 2024-04-26T01:55:23.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.086+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c018680 con 0x7f7fa006b0f0 2024-04-26T01:55:23.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.086+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a7e10 con 0x7f7fa006b0f0 2024-04-26T01:55:23.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.087+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fa0069730 con 0x7f7fa006b0f0 2024-04-26T01:55:23.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.089+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f7f8c010430 con 0x7f7fa006b0f0 2024-04-26T01:55:23.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.091+0000 7f7f9e000700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7f880a3cd0 0x7f7f880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:23.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.091+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f7f8c0d0c80 con 0x7f7fa006b0f0 2024-04-26T01:55:23.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.091+0000 7f7fa5800700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7f880a3cd0 0x7f7f880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:23.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.092+0000 7f7fa5800700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7f880a3cd0 0x7f7f880a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f940098e0 tx=0x7f7f94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:23.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.094+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f8c09d4b0 con 0x7f7fa006b0f0 2024-04-26T01:55:23.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.326+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7fa002e140 con 0x7f7fa006b0f0 2024-04-26T01:55:23.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.329+0000 7f7f9e000700 1 -- 172.21.15.37:0/1407930997 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22499 (secure 0 0 0) 0x7f7f8c09cdb0 con 0x7f7fa006b0f0 2024-04-26T01:55:23.329 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:55:23.329 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:44.157091+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:55:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.332+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7f880a3cd0 msgr2=0x7f7f880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.332+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7f880a3cd0 0x7f7f880a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7f940098e0 tx=0x7f7f94005960 comp rx=0 tx=0).stop 2024-04-26T01:55:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.332+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 msgr2=0x7f7fa0101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:23.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.332+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 0x7f7fa0101f50 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00e9a0 tx=0x7f7f8c00ecb0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 shutdown_connections 2024-04-26T01:55:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7f880a3cd0 0x7f7f880a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7fa0108a30 0x7f7fa0102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7fa006a750 0x7f7fa0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 --2- 172.21.15.37:0/1407930997 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa006b0f0 0x7f7fa0101f50 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:23.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 >> 172.21.15.37:0/1407930997 conn(0x7f7fa00788f0 msgr2=0x7f7fa00ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:23.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 shutdown_connections 2024-04-26T01:55:23.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:23.333+0000 7f7fa8200700 1 -- 172.21.15.37:0/1407930997 wait complete. 2024-04-26T01:55:24.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:23 smithi037 ceph-mon[27906]: pgmap v371: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:24.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:23 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1407930997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:24.117 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T01:55:24.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:55:24.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:23 smithi045 ceph-mon[32520]: pgmap v371: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:24.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:23 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1407930997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:23 smithi071 ceph-mon[32578]: pgmap v371: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:23 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1407930997' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:24.582 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:26.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:25 smithi037 ceph-mon[27906]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:26.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:25 smithi045 ceph-mon[32520]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:25 smithi071 ceph-mon[32578]: pgmap v372: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:26.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 -- 172.21.15.37:0/1992357457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794069a40 msgr2=0x7f8794105840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:26.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 --2- 172.21.15.37:0/1992357457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794069a40 0x7f8794105840 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8784009990 tx=0x7f8784009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 -- 172.21.15.37:0/1992357457 shutdown_connections 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 --2- 172.21.15.37:0/1992357457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794069a40 0x7f8794105840 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 --2- 172.21.15.37:0/1992357457 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f879410bae0 0x7f879410ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 --2- 172.21.15.37:0/1992357457 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8794101960 0x7f8794105d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 -- 172.21.15.37:0/1992357457 >> 172.21.15.37:0/1992357457 conn(0x7f87940766e0 msgr2=0x7f8794076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 -- 172.21.15.37:0/1992357457 shutdown_connections 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.943+0000 7f8793e00700 1 -- 172.21.15.37:0/1992357457 wait complete. 2024-04-26T01:55:26.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.944+0000 7f8793e00700 1 Processor -- start 2024-04-26T01:55:26.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 -- start start 2024-04-26T01:55:26.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 0x7f8794198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:26.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794101960 0x7f8794199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f879410bae0 0x7f879419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8794110960 con 0x7f8794101960 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f87941107e0 con 0x7f879410bae0 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8794110ae0 con 0x7f8794069a40 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8792a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 0x7f8794198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8792000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794101960 0x7f8794199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8792a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 0x7f8794198fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34058/0 (socket says 172.21.15.37:34058) 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8792a00700 1 -- 172.21.15.37:0/4107070038 learned_addr learned my addr 172.21.15.37:0/4107070038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:26.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.945+0000 7f8793400700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f879410bae0 0x7f879419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8792a00700 1 -- 172.21.15.37:0/4107070038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f879410bae0 msgr2=0x7f879419d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8792a00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f879410bae0 0x7f879419d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8792a00700 1 -- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794101960 msgr2=0x7f8794199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8792a00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794101960 0x7f8794199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8792a00700 1 -- 172.21.15.37:0/4107070038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f879419dfe0 con 0x7f8794069a40 2024-04-26T01:55:26.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8793400700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f879410bae0 0x7f879419d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.946+0000 7f8792a00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 0x7f8794198fd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8784005910 tx=0x7f878400b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.947+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784027070 con 0x7f8794069a40 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.947+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8784022470 con 0x7f8794069a40 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.947+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f879419e270 con 0x7f8794069a40 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.947+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784017650 con 0x7f8794069a40 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.947+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87941a5b50 con 0x7f8794069a40 2024-04-26T01:55:26.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.948+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8794069730 con 0x7f8794069a40 2024-04-26T01:55:26.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.950+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f878402a070 con 0x7f8794069a40 2024-04-26T01:55:26.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.951+0000 7f878b400700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f87780a3cd0 0x7f87780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.952+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f87840d1260 con 0x7f8794069a40 2024-04-26T01:55:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.952+0000 7f8792000700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f87780a3cd0 0x7f87780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:26.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.953+0000 7f8792000700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f87780a3cd0 0x7f87780a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8794100610 tx=0x7f8780010040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:26.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:26.955+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8784013070 con 0x7f8794069a40 2024-04-26T01:55:27.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.188+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87941a5d20 con 0x7f8794069a40 2024-04-26T01:55:27.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.190+0000 7f878b400700 1 -- 172.21.15.37:0/4107070038 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22499 (secure 0 0 0) 0x7f878409db20 con 0x7f8794069a40 2024-04-26T01:55:27.190 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:55:27.190 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:44.157091+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:55:27.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f87780a3cd0 msgr2=0x7f87780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f87780a3cd0 0x7f87780a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8794100610 tx=0x7f8780010040 comp rx=0 tx=0).stop 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 msgr2=0x7f8794198fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 0x7f8794198fd0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8784005910 tx=0x7f878400b670 comp rx=0 tx=0).stop 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 shutdown_connections 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f87780a3cd0 0x7f87780a6190 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f879410bae0 0x7f879419d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8794069a40 0x7f8794198fd0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 --2- 172.21.15.37:0/4107070038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8794101960 0x7f8794199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 >> 172.21.15.37:0/4107070038 conn(0x7f87940766e0 msgr2=0x7f879410c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.193+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 shutdown_connections 2024-04-26T01:55:27.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:27.194+0000 7f8793e00700 1 -- 172.21.15.37:0/4107070038 wait complete. 2024-04-26T01:55:28.116 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T01:55:28.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T01:55:28.117 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T01:55:28.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T01:55:28.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T01:55:28.118 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T01:55:28.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T01:55:28.119 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T01:55:28.120 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T01:55:28.120 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T01:55:28.121 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T01:55:28.121 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T01:55:28.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:27 smithi037 ceph-mon[27906]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:28.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:27 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4107070038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:27 smithi045 ceph-mon[32520]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:27 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4107070038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:27 smithi071 ceph-mon[32578]: pgmap v373: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:27 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4107070038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:55:29.155 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.580 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.584 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.592 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.593 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.604 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.611 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.612 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.617 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:30.123 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:30.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:29 smithi037 ceph-mon[27906]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:29 smithi045 ceph-mon[32520]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:29 smithi071 ceph-mon[32578]: pgmap v374: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:31 smithi045 ceph-mon[32520]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:32.323 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:31 smithi037 ceph-mon[27906]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:31 smithi071 ceph-mon[32578]: pgmap v375: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:34.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:33 smithi037 ceph-mon[27906]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:33 smithi045 ceph-mon[32520]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:33 smithi071 ceph-mon[32578]: pgmap v376: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:36.090 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:35 smithi037 ceph-mon[27906]: pgmap v377: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:35 smithi045 ceph-mon[32520]: pgmap v377: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:35 smithi071 ceph-mon[32578]: pgmap v377: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:37 smithi045 ceph-mon[32520]: pgmap v378: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:38.376 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:37 smithi037 ceph-mon[27906]: pgmap v378: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:37 smithi071 ceph-mon[32578]: pgmap v378: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:40.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:39 smithi037 ceph-mon[27906]: pgmap v379: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:40.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:39 smithi045 ceph-mon[32520]: pgmap v379: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:39 smithi071 ceph-mon[32578]: pgmap v379: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:41 smithi037 ceph-mon[27906]: pgmap v380: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:41 smithi045 ceph-mon[32520]: pgmap v380: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:41 smithi071 ceph-mon[32578]: pgmap v380: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:44.134 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:44 smithi037 ceph-mon[27906]: pgmap v381: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.205+0000 7f614b000700 1 -- 172.21.15.37:0/3825020947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144101700 msgr2=0x7f614410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.205+0000 7f614b000700 1 --2- 172.21.15.37:0/3825020947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144101700 0x7f614410ede0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6130009990 tx=0x7f6130009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 -- 172.21.15.37:0/3825020947 shutdown_connections 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 --2- 172.21.15.37:0/3825020947 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f614410f320 0x7f6144111710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 --2- 172.21.15.37:0/3825020947 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6144100de0 0x7f61441011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 --2- 172.21.15.37:0/3825020947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144101700 0x7f614410ede0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 -- 172.21.15.37:0/3825020947 >> 172.21.15.37:0/3825020947 conn(0x7f61440766e0 msgr2=0x7f6144076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 -- 172.21.15.37:0/3825020947 shutdown_connections 2024-04-26T01:55:44.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.206+0000 7f614b000700 1 -- 172.21.15.37:0/3825020947 wait complete. 2024-04-26T01:55:44.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.207+0000 7f614b000700 1 Processor -- start 2024-04-26T01:55:44.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.207+0000 7f614b000700 1 -- start start 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614b000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144100de0 0x7f614419b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614b000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6144101700 0x7f614419b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614b000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f614410f320 0x7f614419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614b000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6144114d80 con 0x7f6144100de0 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614b000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6144114c00 con 0x7f6144101700 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614b000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6144114f00 con 0x7f614410f320 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144100de0 0x7f614419b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:44.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6144101700 0x7f614419b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:44.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144100de0 0x7f614419b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58894/0 (socket says 172.21.15.37:58894) 2024-04-26T01:55:44.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149c00700 1 -- 172.21.15.37:0/3183025180 learned_addr learned my addr 172.21.15.37:0/3183025180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:44.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f614a600700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f614410f320 0x7f614419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:44.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149200700 1 -- 172.21.15.37:0/3183025180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f614410f320 msgr2=0x7f614419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149200700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f614410f320 0x7f614419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.208+0000 7f6149200700 1 -- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144100de0 msgr2=0x7f614419b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6149200700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144100de0 0x7f614419b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6149200700 1 -- 172.21.15.37:0/3183025180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441a02a0 con 0x7f6144101700 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6149c00700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6144100de0 0x7f614419b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6149200700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6144101700 0x7f614419b7d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6130009990 tx=0x7f61300047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130027070 con 0x7f6144101700 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61441a0530 con 0x7f6144101700 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6130022470 con 0x7f6144101700 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130017650 con 0x7f6144101700 2024-04-26T01:55:44.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.209+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61441a7e10 con 0x7f6144101700 2024-04-26T01:55:44.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.210+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61440631d0 con 0x7f6144101700 2024-04-26T01:55:44.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.213+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f613002a070 con 0x7f6144101700 2024-04-26T01:55:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.214+0000 7f6142000700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f612c0a3cd0 0x7f612c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.214+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f61300d1990 con 0x7f6144101700 2024-04-26T01:55:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.214+0000 7f6149c00700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f612c0a3cd0 0x7f612c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:44.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.214+0000 7f6142000700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] conn(0x7f612c0ad810 0x7f612c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.215+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 --> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6130011ec0 con 0x7f612c0ad810 2024-04-26T01:55:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.215+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f61300d1d40 con 0x7f6144101700 2024-04-26T01:55:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.215+0000 7f614a600700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] conn(0x7f612c0ad810 0x7f612c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.215+0000 7f6149c00700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f612c0a3cd0 0x7f612c0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6138011590 tx=0x7f6138009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:44.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.215+0000 7f614a600700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] conn(0x7f612c0ad810 0x7f612c0afcb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:44.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.216+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== osd.3 v2:172.21.15.37:6810/1364499225 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6130011ec0 con 0x7f612c0ad810 2024-04-26T01:55:44.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.235+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 --> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f61440635d0 con 0x7f612c0ad810 2024-04-26T01:55:44.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.236+0000 7f6142000700 1 -- 172.21.15.37:0/3183025180 <== osd.3 v2:172.21.15.37:6810/1364499225 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f61440635d0 con 0x7f612c0ad810 2024-04-26T01:55:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.236+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] conn(0x7f612c0ad810 msgr2=0x7f612c0afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.236+0000 7f614b000700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] conn(0x7f612c0ad810 0x7f612c0afcb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f612c0a3cd0 msgr2=0x7f612c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:44.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f612c0a3cd0 0x7f612c0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6138011590 tx=0x7f6138009260 comp rx=0 tx=0).stop 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6144101700 msgr2=0x7f614419b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 --2- 172.21.15.37:0/3183025180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6144101700 0x7f614419b7d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6130009990 tx=0x7f61300047b0 comp rx=0 tx=0).stop 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614a600700 1 -- 172.21.15.37:0/3183025180 reap_dead start 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 shutdown_connections 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 >> 172.21.15.37:0/3183025180 conn(0x7f61440766e0 msgr2=0x7f614410f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 shutdown_connections 2024-04-26T01:55:44.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.237+0000 7f614b000700 1 -- 172.21.15.37:0/3183025180 wait complete. 2024-04-26T01:55:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:44 smithi045 ceph-mon[32520]: pgmap v381: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:44 smithi071 ceph-mon[32578]: pgmap v381: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 -- 172.21.15.37:0/371835039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a8075590 msgr2=0x7fe9a810ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 --2- 172.21.15.37:0/371835039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a8075590 0x7fe9a810ef60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe994009990 tx=0x7fe994009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 -- 172.21.15.37:0/371835039 shutdown_connections 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 --2- 172.21.15.37:0/371835039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9a810f4a0 0x7fe9a8111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 --2- 172.21.15.37:0/371835039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9a8074c70 0x7fe9a8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 --2- 172.21.15.37:0/371835039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a8075590 0x7fe9a810ef60 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 -- 172.21.15.37:0/371835039 >> 172.21.15.37:0/371835039 conn(0x7fe9a8079ba0 msgr2=0x7fe9a8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 -- 172.21.15.37:0/371835039 shutdown_connections 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.848+0000 7fe9ad000700 1 -- 172.21.15.37:0/371835039 wait complete. 2024-04-26T01:55:45.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 Processor -- start 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 -- start start 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9a8074c70 0x7fe9a819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9a8075590 0x7fe9a819b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a810f4a0 0x7fe9a819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8114eb0 con 0x7fe9a810f4a0 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8114d30 con 0x7fe9a8074c70 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.849+0000 7fe9ad000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a8115030 con 0x7fe9a8075590 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a810f4a0 0x7fe9a819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a810f4a0 0x7fe9a819fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58882/0 (socket says 172.21.15.37:58882) 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 -- 172.21.15.37:0/242489376 learned_addr learned my addr 172.21.15.37:0/242489376 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a5e00700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9a8074c70 0x7fe9a819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a5400700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9a8075590 0x7fe9a819b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 -- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9a8075590 msgr2=0x7fe9a819b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9a8075590 0x7fe9a819b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 -- 172.21.15.37:0/242489376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9a8074c70 msgr2=0x7fe9a819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe9a8074c70 0x7fe9a819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 -- 172.21.15.37:0/242489376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a81a0310 con 0x7fe9a810f4a0 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a5400700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe9a8075590 0x7fe9a819b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.850+0000 7fe9a6800700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a810f4a0 0x7fe9a819fbd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000efc0 tx=0x7fe9a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.851+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a00092e0 con 0x7fe9a810f4a0 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.851+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a81a05a0 con 0x7fe9a810f4a0 2024-04-26T01:55:45.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.851+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9a0004bb0 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.851+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a0004280 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.851+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a81a7e80 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.852+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe9a8100650 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.854+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe9a001b030 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.855+0000 7fe99ea00700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe98c0a3cd0 0x7fe98c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.856+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe9a00d0a00 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.856+0000 7fe9a5e00700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe98c0a3cd0 0x7fe98c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.856+0000 7fe99ea00700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] conn(0x7fe98c0ad810 0x7fe98c0afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.856+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 --> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9a00048a0 con 0x7fe98c0ad810 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.856+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe9a00d0db0 con 0x7fe9a810f4a0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.856+0000 7fe9a5400700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] conn(0x7fe98c0ad810 0x7fe98c0afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.857+0000 7fe9a5e00700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe98c0a3cd0 0x7fe98c0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe998009c60 tx=0x7fe998009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.857+0000 7fe9a5400700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] conn(0x7fe98c0ad810 0x7fe98c0afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:45.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.858+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== osd.2 v2:172.21.15.71:6800/2117420904 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9a00048a0 con 0x7fe98c0ad810 2024-04-26T01:55:45.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.877+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 --> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe9a8076330 con 0x7fe98c0ad810 2024-04-26T01:55:45.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe99ea00700 1 -- 172.21.15.37:0/242489376 <== osd.2 v2:172.21.15.71:6800/2117420904 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe9a8076330 con 0x7fe98c0ad810 2024-04-26T01:55:45.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] conn(0x7fe98c0ad810 msgr2=0x7fe98c0afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] conn(0x7fe98c0ad810 0x7fe98c0afcb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe98c0a3cd0 msgr2=0x7fe98c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe98c0a3cd0 0x7fe98c0a6190 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe998009c60 tx=0x7fe998009260 comp rx=0 tx=0).stop 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a810f4a0 msgr2=0x7fe9a819fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 --2- 172.21.15.37:0/242489376 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe9a810f4a0 0x7fe9a819fbd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe9a000efc0 tx=0x7fe9a000c3a0 comp rx=0 tx=0).stop 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9a6800700 1 -- 172.21.15.37:0/242489376 reap_dead start 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.878+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 shutdown_connections 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.879+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 >> 172.21.15.37:0/242489376 conn(0x7fe9a8079ba0 msgr2=0x7fe9a80ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.879+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 shutdown_connections 2024-04-26T01:55:45.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:43.879+0000 7fe9ad000700 1 -- 172.21.15.37:0/242489376 wait complete. 2024-04-26T01:55:46.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:46 smithi037 ceph-mon[27906]: pgmap v382: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:46 smithi071 ceph-mon[32578]: pgmap v382: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:46 smithi045 ceph-mon[32520]: pgmap v382: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:55:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 -- 172.21.15.37:0/3519202081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006a750 msgr2=0x7f700006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 --2- 172.21.15.37:0/3519202081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006a750 0x7f700006abb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8009990 tx=0x7f6fe8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 -- 172.21.15.37:0/3519202081 shutdown_connections 2024-04-26T01:55:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 --2- 172.21.15.37:0/3519202081 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006b0f0 0x7f700010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 --2- 172.21.15.37:0/3519202081 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7000108a30 0x7f7000108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 --2- 172.21.15.37:0/3519202081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006a750 0x7f700006abb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.368+0000 7f7007a00700 1 -- 172.21.15.37:0/3519202081 >> 172.21.15.37:0/3519202081 conn(0x7f70000788f0 msgr2=0x7f7000078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.369+0000 7f7007a00700 1 -- 172.21.15.37:0/3519202081 shutdown_connections 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.369+0000 7f7007a00700 1 -- 172.21.15.37:0/3519202081 wait complete. 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.370+0000 7f7007a00700 1 Processor -- start 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.370+0000 7f7007a00700 1 -- start start 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7007a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006a750 0x7f7000101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7007a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006b0f0 0x7f7000101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:46.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7007a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7000108a30 0x7f7000102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7007a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7000112c20 con 0x7f700006b0f0 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7007a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7000112aa0 con 0x7f700006a750 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7007a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7000112da0 con 0x7f7000108a30 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f6ff7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006b0f0 0x7f7000101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7005000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006a750 0x7f7000101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7005000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006a750 0x7f7000101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:42222/0 (socket says 172.21.15.37:42222) 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7005000700 1 -- 172.21.15.37:0/4186211397 learned_addr learned my addr 172.21.15.37:0/4186211397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.371+0000 7f7005a00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7000108a30 0x7f7000102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7005000700 1 -- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7000108a30 msgr2=0x7f7000102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:46.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7005000700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7000108a30 0x7f7000102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7005000700 1 -- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006b0f0 msgr2=0x7f7000101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7005000700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006b0f0 0x7f7000101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7005000700 1 -- 172.21.15.37:0/4186211397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70001a76e0 con 0x7f700006a750 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f6ff7e00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f700006b0f0 0x7f7000101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7005000700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006a750 0x7f7000101a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000eb30 tx=0x7f6ff000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff000cbd0 con 0x7f700006a750 2024-04-26T01:55:46.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.372+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70001a79d0 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.373+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff00044d0 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.373+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0003e40 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.373+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70001a7f60 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.373+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7000063110 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.376+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ff0004000 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.377+0000 7f6ffe000700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6fe40a3d50 0x7f6fe40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.377+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f6ff00d0950 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.377+0000 7f6ff7e00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6fe40a3d50 0x7f6fe40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.378+0000 7f6ffe000700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] conn(0x7f6fe40ad890 0x7f6fe40afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.378+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 --> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ff00090b0 con 0x7f6fe40ad890 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.378+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6ff00d0d00 con 0x7f700006a750 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.378+0000 7f7005a00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] conn(0x7f6fe40ad890 0x7f6fe40afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.378+0000 7f6ff7e00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6fe40a3d50 0x7f6fe40a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7000078f60 tx=0x7f6fe800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:46.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.379+0000 7f7005a00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] conn(0x7f6fe40ad890 0x7f6fe40afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.380+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== osd.8 v2:172.21.15.71:6816/1870194976 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ff00090b0 con 0x7f6fe40ad890 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.398+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 --> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7000063590 con 0x7f6fe40ad890 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.398+0000 7f6ffe000700 1 -- 172.21.15.37:0/4186211397 <== osd.8 v2:172.21.15.71:6816/1870194976 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7000063590 con 0x7f6fe40ad890 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.399+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] conn(0x7f6fe40ad890 msgr2=0x7f6fe40afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.399+0000 7f7007a00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] conn(0x7f6fe40ad890 0x7f6fe40afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6fe40a3d50 msgr2=0x7f6fe40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6fe40a3d50 0x7f6fe40a6210 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7000078f60 tx=0x7f6fe800b260 comp rx=0 tx=0).stop 2024-04-26T01:55:46.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006a750 msgr2=0x7f7000101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 --2- 172.21.15.37:0/4186211397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f700006a750 0x7f7000101a10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000eb30 tx=0x7f6ff000ee40 comp rx=0 tx=0).stop 2024-04-26T01:55:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7005a00700 1 -- 172.21.15.37:0/4186211397 reap_dead start 2024-04-26T01:55:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 shutdown_connections 2024-04-26T01:55:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 >> 172.21.15.37:0/4186211397 conn(0x7f70000788f0 msgr2=0x7f70000ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 shutdown_connections 2024-04-26T01:55:46.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:45.400+0000 7f7007a00700 1 -- 172.21.15.37:0/4186211397 wait complete. 2024-04-26T01:55:47.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.793+0000 7f102b400700 1 -- 172.21.15.37:0/2824682576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406a750 msgr2=0x7f102406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.793+0000 7f102b400700 1 --2- 172.21.15.37:0/2824682576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406a750 0x7f102406abb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f1010009990 tx=0x7f1010009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 -- 172.21.15.37:0/2824682576 shutdown_connections 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 --2- 172.21.15.37:0/2824682576 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406b0f0 0x7f102410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 --2- 172.21.15.37:0/2824682576 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024108a30 0x7f1024108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 --2- 172.21.15.37:0/2824682576 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406a750 0x7f102406abb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 -- 172.21.15.37:0/2824682576 >> 172.21.15.37:0/2824682576 conn(0x7f10240788f0 msgr2=0x7f1024078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 -- 172.21.15.37:0/2824682576 shutdown_connections 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.794+0000 7f102b400700 1 -- 172.21.15.37:0/2824682576 wait complete. 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 Processor -- start 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 -- start start 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406a750 0x7f102419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:47.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406b0f0 0x7f102419b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024108a30 0x7f102419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1024112c20 con 0x7f102406b0f0 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1024112aa0 con 0x7f102406a750 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.795+0000 7f102b400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1024112da0 con 0x7f1024108a30 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1023e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406b0f0 0x7f102419b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406a750 0x7f102419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1023e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406b0f0 0x7f102419b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58900/0 (socket says 172.21.15.37:58900) 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1023e00700 1 -- 172.21.15.37:0/3256176317 learned_addr learned my addr 172.21.15.37:0/3256176317 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406a750 0x7f102419b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:42202/0 (socket says 172.21.15.37:42202) 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1029400700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024108a30 0x7f102419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:47.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 -- 172.21.15.37:0/3256176317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024108a30 msgr2=0x7f102419fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1024108a30 0x7f102419fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 -- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406b0f0 msgr2=0x7f102419b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406b0f0 0x7f102419b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1028a00700 1 -- 172.21.15.37:0/3256176317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10241a0390 con 0x7f102406a750 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.796+0000 7f1023e00700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f102406b0f0 0x7f102419b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.797+0000 7f1028a00700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406a750 0x7f102419b380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f101800e9f0 tx=0x7f101800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.797+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101800cb70 con 0x7f102406a750 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.797+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f101800ee80 con 0x7f102406a750 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.797+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1018003d40 con 0x7f102406a750 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.797+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10241a0680 con 0x7f102406a750 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.797+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10241a7f00 con 0x7f102406a750 2024-04-26T01:55:47.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.799+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1024063150 con 0x7f102406a750 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.801+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1018003ee0 con 0x7f102406a750 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.802+0000 7f1021600700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f100c0a3d50 0x7f100c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.802+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f10180d05f0 con 0x7f102406a750 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.802+0000 7f1023e00700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f100c0a3d50 0x7f100c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.803+0000 7f1021600700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] conn(0x7f100c0ad890 0x7f100c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.803+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 --> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f100c0b0400 con 0x7f100c0ad890 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.803+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f10180d7050 con 0x7f102406a750 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.803+0000 7f1029400700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] conn(0x7f100c0ad890 0x7f100c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.803+0000 7f1023e00700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f100c0a3d50 0x7f100c0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1024078f40 tx=0x7f101000b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.803+0000 7f1029400700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] conn(0x7f100c0ad890 0x7f100c0afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.804+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== osd.0 v2:172.21.15.37:6802/2603696107 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f100c0b0400 con 0x7f100c0ad890 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.823+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 --> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1024063620 con 0x7f100c0ad890 2024-04-26T01:55:47.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.823+0000 7f1021600700 1 -- 172.21.15.37:0/3256176317 <== osd.0 v2:172.21.15.37:6802/2603696107 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1024063620 con 0x7f100c0ad890 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.823+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] conn(0x7f100c0ad890 msgr2=0x7f100c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.823+0000 7f102b400700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] conn(0x7f100c0ad890 0x7f100c0afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f100c0a3d50 msgr2=0x7f100c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f100c0a3d50 0x7f100c0a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1024078f40 tx=0x7f101000b2a0 comp rx=0 tx=0).stop 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406a750 msgr2=0x7f102419b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 --2- 172.21.15.37:0/3256176317 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f102406a750 0x7f102419b380 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f101800e9f0 tx=0x7f101800ed00 comp rx=0 tx=0).stop 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f1029400700 1 -- 172.21.15.37:0/3256176317 reap_dead start 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 shutdown_connections 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 >> 172.21.15.37:0/3256176317 conn(0x7f10240788f0 msgr2=0x7f102410dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 shutdown_connections 2024-04-26T01:55:47.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:44.824+0000 7f102b400700 1 -- 172.21.15.37:0/3256176317 wait complete. 2024-04-26T01:55:48.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:48 smithi037 ceph-mon[27906]: pgmap v383: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:48 smithi071 ceph-mon[32578]: pgmap v383: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:48.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 -- 172.21.15.37:0/79281053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0804108a30 msgr2=0x7f0804108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 --2- 172.21.15.37:0/79281053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0804108a30 0x7f0804108e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f07f40098f0 tx=0x7f07f4009c00 comp rx=0 tx=0).stop 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 -- 172.21.15.37:0/79281053 shutdown_connections 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 --2- 172.21.15.37:0/79281053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f080406b0f0 0x7f080410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 --2- 172.21.15.37:0/79281053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0804108a30 0x7f0804108e10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 --2- 172.21.15.37:0/79281053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f080406a750 0x7f080406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 -- 172.21.15.37:0/79281053 >> 172.21.15.37:0/79281053 conn(0x7f08040788f0 msgr2=0x7f0804078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 -- 172.21.15.37:0/79281053 shutdown_connections 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.995+0000 7f0809200700 1 -- 172.21.15.37:0/79281053 wait complete. 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.996+0000 7f0809200700 1 Processor -- start 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.996+0000 7f0809200700 1 -- start start 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0809200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f080406a750 0x7f0804199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0809200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f080406b0f0 0x7f08041996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:48.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0809200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0804108a30 0x7f080419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0809200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0804112c00 con 0x7f0804108a30 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0809200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0804112a80 con 0x7f080406a750 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0809200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0804112d80 con 0x7f080406b0f0 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0804108a30 0x7f080419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0804108a30 0x7f080419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58926/0 (socket says 172.21.15.37:58926) 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 -- 172.21.15.37:0/3855975703 learned_addr learned my addr 172.21.15.37:0/3855975703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0801e00700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f080406a750 0x7f0804199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f07f9400700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f080406b0f0 0x7f08041996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 -- 172.21.15.37:0/3855975703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f080406b0f0 msgr2=0x7f08041996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f080406b0f0 0x7f08041996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 -- 172.21.15.37:0/3855975703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f080406a750 msgr2=0x7f0804199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f080406a750 0x7f0804199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0802800700 1 -- 172.21.15.37:0/3855975703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080419e1a0 con 0x7f0804108a30 2024-04-26T01:55:48.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.997+0000 7f0801e00700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f080406a750 0x7f0804199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.998+0000 7f0802800700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0804108a30 0x7f080419da60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c110 tx=0x7f07fc00c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.998+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0092e0 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.998+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07fc004bb0 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.998+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc007be0 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.998+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080419e430 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.998+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08041a5d10 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:46.999+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08040631d0 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.002+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f07fc01a020 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.003+0000 7f07fb400700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07e80a3d50 0x7f07e80a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.003+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f07fc0d0a30 con 0x7f0804108a30 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.003+0000 7f0801e00700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07e80a3d50 0x7f07e80a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.004+0000 7f07fb400700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] conn(0x7f07e80ad890 0x7f07e80afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.004+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 --> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f07fc014e50 con 0x7f07e80ad890 2024-04-26T01:55:48.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.004+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f07fc0d0de0 con 0x7f0804108a30 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.004+0000 7f07f9400700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] conn(0x7f07e80ad890 0x7f07e80afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.004+0000 7f0801e00700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07e80a3d50 0x7f07e80a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f07f40098c0 tx=0x7f07f4019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.005+0000 7f07f9400700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] conn(0x7f07e80ad890 0x7f07e80afd30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.006+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== osd.6 v2:172.21.15.37:6818/2087418626 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f07fc014e50 con 0x7f07e80ad890 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.024+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 --> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f0804063650 con 0x7f07e80ad890 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.025+0000 7f07fb400700 1 -- 172.21.15.37:0/3855975703 <== osd.6 v2:172.21.15.37:6818/2087418626 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f0804063650 con 0x7f07e80ad890 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] conn(0x7f07e80ad890 msgr2=0x7f07e80afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:48.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] conn(0x7f07e80ad890 0x7f07e80afd30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07e80a3d50 msgr2=0x7f07e80a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07e80a3d50 0x7f07e80a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f07f40098c0 tx=0x7f07f4019040 comp rx=0 tx=0).stop 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0804108a30 msgr2=0x7f080419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 --2- 172.21.15.37:0/3855975703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0804108a30 0x7f080419da60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f07fc00c110 tx=0x7f07fc00c420 comp rx=0 tx=0).stop 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0802800700 1 -- 172.21.15.37:0/3855975703 reap_dead start 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 shutdown_connections 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 >> 172.21.15.37:0/3855975703 conn(0x7f08040788f0 msgr2=0x7f08040ffb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 shutdown_connections 2024-04-26T01:55:48.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:47.026+0000 7f0809200700 1 -- 172.21.15.37:0/3855975703 wait complete. 2024-04-26T01:55:48.508 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:48 smithi045 ceph-mon[32520]: pgmap v383: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.740+0000 7f074ca00700 1 -- 172.21.15.37:0/1304724102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748108a80 msgr2=0x7f0748108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.740+0000 7f074ca00700 1 --2- 172.21.15.37:0/1304724102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748108a80 0x7f0748108e60 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0738009990 tx=0x7f0738009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 -- 172.21.15.37:0/1304724102 shutdown_connections 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 --2- 172.21.15.37:0/1304724102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07481034b0 0x7f074810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 --2- 172.21.15.37:0/1304724102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748108a80 0x7f0748108e60 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 --2- 172.21.15.37:0/1304724102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0748102af0 0x7f0748102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 -- 172.21.15.37:0/1304724102 >> 172.21.15.37:0/1304724102 conn(0x7f0748079ad0 msgr2=0x7f0748079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 -- 172.21.15.37:0/1304724102 shutdown_connections 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.741+0000 7f074ca00700 1 -- 172.21.15.37:0/1304724102 wait complete. 2024-04-26T01:55:49.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.742+0000 7f074ca00700 1 Processor -- start 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.742+0000 7f074ca00700 1 -- start start 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f074ca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748102af0 0x7f074819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f074ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07481034b0 0x7f074819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f074ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0748108a80 0x7f074819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f074ca00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0748112c90 con 0x7f0748108a80 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f074ca00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0748112b10 con 0x7f0748102af0 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f074ca00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0748112e10 con 0x7f07481034b0 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f0745e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748102af0 0x7f074819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f0745e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748102af0 0x7f074819b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:42278/0 (socket says 172.21.15.37:42278) 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f0745e00700 1 -- 172.21.15.37:0/1528428709 learned_addr learned my addr 172.21.15.37:0/1528428709 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.743+0000 7f0745400700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07481034b0 0x7f074819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0746800700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0748108a80 0x7f074819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:49.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0745e00700 1 -- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07481034b0 msgr2=0x7f074819b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0745e00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f07481034b0 0x7f074819b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0745e00700 1 -- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0748108a80 msgr2=0x7f074819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0745e00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0748108a80 0x7f074819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0745e00700 1 -- 172.21.15.37:0/1528428709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07481a0370 con 0x7f0748102af0 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0746800700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0748108a80 0x7f074819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f0745e00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748102af0 0x7f074819b360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0738000c00 tx=0x7f07380057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.744+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738027070 con 0x7f0748102af0 2024-04-26T01:55:49.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.745+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07481a0600 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.745+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0738022470 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.745+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0738017650 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.745+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07481a7ee0 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.746+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f074810f8b0 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.748+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f073802a070 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.749+0000 7f073ea00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f072c0a3cd0 0x7f072c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.750+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f07380d1510 con 0x7f0748102af0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.750+0000 7f0745400700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f072c0a3cd0 0x7f072c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.750+0000 7f073ea00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] conn(0x7f072c0ad6e0 0x7f072c0afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:49.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.750+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 --> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0738019e50 con 0x7f072c0ad6e0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.750+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f07380d18c0 con 0x7f0748102af0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.750+0000 7f0746800700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] conn(0x7f072c0ad6e0 0x7f072c0afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.751+0000 7f0745400700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f072c0a3cd0 0x7f072c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f074807a450 tx=0x7f0734009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.751+0000 7f0746800700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] conn(0x7f072c0ad6e0 0x7f072c0afb80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.752+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== osd.5 v2:172.21.15.71:6808/1060444370 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0738019e50 con 0x7f072c0ad6e0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.771+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 --> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f07481041b0 con 0x7f072c0ad6e0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.771+0000 7f073ea00700 1 -- 172.21.15.37:0/1528428709 <== osd.5 v2:172.21.15.71:6808/1060444370 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f07481041b0 con 0x7f072c0ad6e0 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] conn(0x7f072c0ad6e0 msgr2=0x7f072c0afb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] conn(0x7f072c0ad6e0 0x7f072c0afb80 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f072c0a3cd0 msgr2=0x7f072c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f072c0a3cd0 0x7f072c0a6190 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f074807a450 tx=0x7f0734009260 comp rx=0 tx=0).stop 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748102af0 msgr2=0x7f074819b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:49.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 --2- 172.21.15.37:0/1528428709 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0748102af0 0x7f074819b360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f0738000c00 tx=0x7f07380057c0 comp rx=0 tx=0).stop 2024-04-26T01:55:49.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f0746800700 1 -- 172.21.15.37:0/1528428709 reap_dead start 2024-04-26T01:55:49.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 shutdown_connections 2024-04-26T01:55:49.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 >> 172.21.15.37:0/1528428709 conn(0x7f0748079ad0 msgr2=0x7f074810dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:49.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 shutdown_connections 2024-04-26T01:55:49.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.772+0000 7f074ca00700 1 -- 172.21.15.37:0/1528428709 wait complete. 2024-04-26T01:55:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 -- 172.21.15.37:0/101219226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88107990 msgr2=0x7f9b88107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 --2- 172.21.15.37:0/101219226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88107990 0x7f9b88107d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c009990 tx=0x7f9b7c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 -- 172.21.15.37:0/101219226 shutdown_connections 2024-04-26T01:55:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 --2- 172.21.15.37:0/101219226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88107990 0x7f9b88107d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 --2- 172.21.15.37:0/101219226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88102300 0x7f9b8810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 --2- 172.21.15.37:0/101219226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b88101940 0x7f9b88101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 -- 172.21.15.37:0/101219226 >> 172.21.15.37:0/101219226 conn(0x7f9b880788f0 msgr2=0x7f9b88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.532+0000 7f9b90000700 1 -- 172.21.15.37:0/101219226 shutdown_connections 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.533+0000 7f9b90000700 1 -- 172.21.15.37:0/101219226 wait complete. 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.534+0000 7f9b90000700 1 Processor -- start 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.534+0000 7f9b90000700 1 -- start start 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b90000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88101940 0x7f9b881991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b90000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88102300 0x7f9b881996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b90000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b88107990 0x7f9b8819da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b90000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88111b30 con 0x7f9b88102300 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b90000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9b881119b0 con 0x7f9b88107990 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b90000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9b88111cb0 con 0x7f9b88101940 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b8cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88102300 0x7f9b881996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b8cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88102300 0x7f9b881996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58954/0 (socket says 172.21.15.37:58954) 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b8cc00700 1 -- 172.21.15.37:0/2885108936 learned_addr learned my addr 172.21.15.37:0/2885108936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b8d600700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88101940 0x7f9b881991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.535+0000 7f9b8e000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b88107990 0x7f9b8819da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b8d600700 1 -- 172.21.15.37:0/2885108936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b88107990 msgr2=0x7f9b8819da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b8d600700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9b88107990 0x7f9b8819da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b8d600700 1 -- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88102300 msgr2=0x7f9b881996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b8d600700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9b88102300 0x7f9b881996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b8d600700 1 -- 172.21.15.37:0/2885108936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8819e1c0 con 0x7f9b88101940 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b8d600700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88101940 0x7f9b881991b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0098e0 tx=0x7f9b7c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c027070 con 0x7f9b88101940 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c022470 con 0x7f9b88101940 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8819e450 con 0x7f9b88101940 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c017650 con 0x7f9b88101940 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.536+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b88103420 con 0x7f9b88101940 2024-04-26T01:55:50.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.537+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b881068d0 con 0x7f9b88101940 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.540+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9b7c02a070 con 0x7f9b88101940 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.541+0000 7f9b86000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9b700a3cd0 0x7f9b700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.541+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f9b7c0d16d0 con 0x7f9b88101940 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.541+0000 7f9b8cc00700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9b700a3cd0 0x7f9b700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.541+0000 7f9b86000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] conn(0x7f9b700ad810 0x7f9b700afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.541+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 --> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b7c001d30 con 0x7f9b700ad810 2024-04-26T01:55:50.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.541+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c0d1a80 con 0x7f9b88101940 2024-04-26T01:55:50.326 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:50 smithi037 ceph-mon[27906]: pgmap v384: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.542+0000 7f9b8e000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] conn(0x7f9b700ad810 0x7f9b700afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.542+0000 7f9b8cc00700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9b700a3cd0 0x7f9b700a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9b88078f30 tx=0x7f9b74009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.542+0000 7f9b8e000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] conn(0x7f9b700ad810 0x7f9b700afcb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.543+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== osd.11 v2:172.21.15.71:6824/584658428 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b7c001d30 con 0x7f9b700ad810 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.561+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 --> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b88103060 con 0x7f9b700ad810 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.562+0000 7f9b86000700 1 -- 172.21.15.37:0/2885108936 <== osd.11 v2:172.21.15.71:6824/584658428 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b88103060 con 0x7f9b700ad810 2024-04-26T01:55:50.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] conn(0x7f9b700ad810 msgr2=0x7f9b700afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] conn(0x7f9b700ad810 0x7f9b700afcb0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9b700a3cd0 msgr2=0x7f9b700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f9b700a3cd0 0x7f9b700a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9b88078f30 tx=0x7f9b74009260 comp rx=0 tx=0).stop 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88101940 msgr2=0x7f9b881991b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 --2- 172.21.15.37:0/2885108936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9b88101940 0x7f9b881991b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0098e0 tx=0x7f9b7c0057c0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b8e000700 1 -- 172.21.15.37:0/2885108936 reap_dead start 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 shutdown_connections 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 >> 172.21.15.37:0/2885108936 conn(0x7f9b880788f0 msgr2=0x7f9b8806a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 shutdown_connections 2024-04-26T01:55:50.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:48.563+0000 7f9b90000700 1 -- 172.21.15.37:0/2885108936 wait complete. 2024-04-26T01:55:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:50 smithi071 ceph-mon[32578]: pgmap v384: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:50 smithi045 ceph-mon[32520]: pgmap v384: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:50.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 -- 172.21.15.37:0/4170332482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c075590 msgr2=0x7fec7c10ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 --2- 172.21.15.37:0/4170332482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c075590 0x7fec7c10ef60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fec680098f0 tx=0x7fec68009c00 comp rx=0 tx=0).stop 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 -- 172.21.15.37:0/4170332482 shutdown_connections 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 --2- 172.21.15.37:0/4170332482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec7c10f4a0 0x7fec7c111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 --2- 172.21.15.37:0/4170332482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c074c70 0x7fec7c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 --2- 172.21.15.37:0/4170332482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c075590 0x7fec7c10ef60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 -- 172.21.15.37:0/4170332482 >> 172.21.15.37:0/4170332482 conn(0x7fec7c079ba0 msgr2=0x7fec7c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.170+0000 7fec82a00700 1 -- 172.21.15.37:0/4170332482 shutdown_connections 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.171+0000 7fec82a00700 1 -- 172.21.15.37:0/4170332482 wait complete. 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.171+0000 7fec82a00700 1 Processor -- start 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 -- start start 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec7c074c70 0x7fec7c102bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c075590 0x7fec7c103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c10f4a0 0x7fec7c103640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c114e40 con 0x7fec7c074c70 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c114cc0 con 0x7fec7c10f4a0 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec82a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec7c114fc0 con 0x7fec7c075590 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec7be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec7c074c70 0x7fec7c102bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec7b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c075590 0x7fec7c103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec7b400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c075590 0x7fec7c103100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45058/0 (socket says 172.21.15.37:45058) 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec7b400700 1 -- 172.21.15.37:0/1406083694 learned_addr learned my addr 172.21.15.37:0/1406083694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.172+0000 7fec7be00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec7c074c70 0x7fec7c102bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58850/0 (socket says 172.21.15.37:58850) 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec80a00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c10f4a0 0x7fec7c103640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec7b400700 1 -- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c10f4a0 msgr2=0x7fec7c103640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec7b400700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec7c10f4a0 0x7fec7c103640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec7b400700 1 -- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec7c074c70 msgr2=0x7fec7c102bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec7b400700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec7c074c70 0x7fec7c102bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec7b400700 1 -- 172.21.15.37:0/1406083694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7c1aba00 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec7b400700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c075590 0x7fec7c103100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec6800f9c0 tx=0x7fec6800f9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec680052d0 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec6800ba10 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec7c1abc90 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.173+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec68004c40 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.174+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec7c1ac140 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.174+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fec7c063150 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.177+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fec68005500 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.178+0000 7fec78c00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fec600a3d50 0x7fec600a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.178+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fec680d1ff0 con 0x7fec7c075590 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.178+0000 7fec7be00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fec600a3d50 0x7fec600a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.178+0000 7fec78c00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] conn(0x7fec600ad890 0x7fec600afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.178+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 --> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fec68018e50 con 0x7fec600ad890 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.178+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fec680d2400 con 0x7fec7c075590 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.179+0000 7fec80a00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] conn(0x7fec600ad890 0x7fec600afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.179+0000 7fec7be00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fec600a3d50 0x7fec600a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009680 tx=0x7fec6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.179+0000 7fec80a00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] conn(0x7fec600ad890 0x7fec600afd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.180+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== osd.1 v2:172.21.15.45:6800/3445601743 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fec68018e50 con 0x7fec600ad890 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.198+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 --> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fec7c0635d0 con 0x7fec600ad890 2024-04-26T01:55:50.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.199+0000 7fec78c00700 1 -- 172.21.15.37:0/1406083694 <== osd.1 v2:172.21.15.45:6800/3445601743 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fec7c0635d0 con 0x7fec600ad890 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.199+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] conn(0x7fec600ad890 msgr2=0x7fec600afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.199+0000 7fec82a00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] conn(0x7fec600ad890 0x7fec600afd30 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fec600a3d50 msgr2=0x7fec600a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fec600a3d50 0x7fec600a6210 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009680 tx=0x7fec6c009260 comp rx=0 tx=0).stop 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c075590 msgr2=0x7fec7c103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 --2- 172.21.15.37:0/1406083694 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec7c075590 0x7fec7c103100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fec6800f9c0 tx=0x7fec6800f9f0 comp rx=0 tx=0).stop 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec80a00700 1 -- 172.21.15.37:0/1406083694 reap_dead start 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 shutdown_connections 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 >> 172.21.15.37:0/1406083694 conn(0x7fec7c079ba0 msgr2=0x7fec7c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 shutdown_connections 2024-04-26T01:55:50.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:41.200+0000 7fec82a00700 1 -- 172.21.15.37:0/1406083694 wait complete. 2024-04-26T01:55:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:52 smithi071 ceph-mon[32578]: pgmap v385: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:52 smithi037 ceph-mon[27906]: pgmap v385: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:52 smithi045 ceph-mon[32520]: pgmap v385: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:52.602 INFO:teuthology.orchestra.run.smithi037.stdout:55834574941 2024-04-26T01:55:52.603 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T01:55:53.206 INFO:teuthology.orchestra.run.smithi037.stdout:347892351016 2024-04-26T01:55:53.207 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T01:55:53.315 INFO:teuthology.orchestra.run.smithi037.stdout:257698037814 2024-04-26T01:55:53.316 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T01:55:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 -- 172.21.15.37:0/1097953322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c102af0 msgr2=0x7f503c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 --2- 172.21.15.37:0/1097953322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c102af0 0x7f503c102f70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f502c009920 tx=0x7f502c009c30 comp rx=0 tx=0).stop 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 -- 172.21.15.37:0/1097953322 shutdown_connections 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 --2- 172.21.15.37:0/1097953322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c1034b0 0x7f503c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 --2- 172.21.15.37:0/1097953322 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f503c107940 0x7f503c107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 --2- 172.21.15.37:0/1097953322 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c102af0 0x7f503c102f70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 -- 172.21.15.37:0/1097953322 >> 172.21.15.37:0/1097953322 conn(0x7f503c079ba0 msgr2=0x7f503c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 -- 172.21.15.37:0/1097953322 shutdown_connections 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.885+0000 7f5043600700 1 -- 172.21.15.37:0/1097953322 wait complete. 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.886+0000 7f5043600700 1 Processor -- start 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 -- start start 2024-04-26T01:55:53.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c102af0 0x7f503c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c1034b0 0x7f503c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f503c107940 0x7f503c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f503c112c30 con 0x7f503c102af0 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f503c112ab0 con 0x7f503c1034b0 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5043600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f503c112db0 con 0x7f503c107940 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5040c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c102af0 0x7f503c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5040c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c102af0 0x7f503c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59012/0 (socket says 172.21.15.37:59012) 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.887+0000 7f5040c00700 1 -- 172.21.15.37:0/147883987 learned_addr learned my addr 172.21.15.37:0/147883987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5041600700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f503c107940 0x7f503c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5033e00700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c1034b0 0x7f503c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5040c00700 1 -- 172.21.15.37:0/147883987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f503c107940 msgr2=0x7f503c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5040c00700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f503c107940 0x7f503c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5040c00700 1 -- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c1034b0 msgr2=0x7f503c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5040c00700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f503c1034b0 0x7f503c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5040c00700 1 -- 172.21.15.37:0/147883987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f503c19e1b0 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f5040c00700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c102af0 0x7f503c1991a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f502400e9c0 tx=0x7f502400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.888+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024004a30 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.889+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f503c19e4a0 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.889+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f502400cc70 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.889+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5024005490 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.889+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f503c1a5d20 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.889+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f503c0631d0 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.892+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5024004bd0 con 0x7f503c102af0 2024-04-26T01:55:53.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.893+0000 7f503a000700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f501c0a3d50 0x7f501c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.893+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f50240d09c0 con 0x7f503c102af0 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.893+0000 7f5033e00700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f501c0a3d50 0x7f501c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.894+0000 7f503a000700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] conn(0x7f501c0ad890 0x7f501c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.894+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 --> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f502401ee50 con 0x7f501c0ad890 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.894+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f50240d0d70 con 0x7f503c102af0 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.894+0000 7f5041600700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] conn(0x7f501c0ad890 0x7f501c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.894+0000 7f5033e00700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f501c0a3d50 0x7f501c0a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f503c07a2e0 tx=0x7f502c019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.895+0000 7f5041600700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] conn(0x7f501c0ad890 0x7f501c0afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.895+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== osd.7 v2:172.21.15.45:6816/1922916284 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f502401ee50 con 0x7f501c0ad890 2024-04-26T01:55:53.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.914+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 --> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f503c0635d0 con 0x7f501c0ad890 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.915+0000 7f503a000700 1 -- 172.21.15.37:0/147883987 <== osd.7 v2:172.21.15.45:6816/1922916284 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f503c0635d0 con 0x7f501c0ad890 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.915+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] conn(0x7f501c0ad890 msgr2=0x7f501c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.915+0000 7f5043600700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] conn(0x7f501c0ad890 0x7f501c0afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f501c0a3d50 msgr2=0x7f501c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f501c0a3d50 0x7f501c0a6210 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f503c07a2e0 tx=0x7f502c019040 comp rx=0 tx=0).stop 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c102af0 msgr2=0x7f503c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 --2- 172.21.15.37:0/147883987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f503c102af0 0x7f503c1991a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f502400e9c0 tx=0x7f502400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5041600700 1 -- 172.21.15.37:0/147883987 reap_dead start 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 shutdown_connections 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 >> 172.21.15.37:0/147883987 conn(0x7f503c079ba0 msgr2=0x7f503c100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:53.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 shutdown_connections 2024-04-26T01:55:53.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:49.916+0000 7f5043600700 1 -- 172.21.15.37:0/147883987 wait complete. 2024-04-26T01:55:54.140 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:54 smithi037 ceph-mon[27906]: pgmap v386: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:54.399 INFO:teuthology.orchestra.run.smithi037.stdout:77309411418 2024-04-26T01:55:54.400 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T01:55:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:54 smithi071 ceph-mon[32578]: pgmap v386: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:54 smithi045 ceph-mon[32520]: pgmap v386: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:55:54.698 INFO:teuthology.orchestra.run.smithi037.stdout:223338299453 2024-04-26T01:55:54.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T01:55:54.987 INFO:teuthology.orchestra.run.smithi037.stdout:111669149778 2024-04-26T01:55:54.987 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 -- 172.21.15.37:0/3633207307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006afe0 msgr2=0x7f28e006b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 --2- 172.21.15.37:0/3633207307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006afe0 0x7f28e006b3c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f28d0009990 tx=0x7f28d0009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 -- 172.21.15.37:0/3633207307 shutdown_connections 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 --2- 172.21.15.37:0/3633207307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006afe0 0x7f28e006b3c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 --2- 172.21.15.37:0/3633207307 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e010f4e0 0x7f28e01118d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 --2- 172.21.15.37:0/3633207307 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006b900 0x7f28e010efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 -- 172.21.15.37:0/3633207307 >> 172.21.15.37:0/3633207307 conn(0x7f28e00788b0 msgr2=0x7f28e0078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 -- 172.21.15.37:0/3633207307 shutdown_connections 2024-04-26T01:55:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.609+0000 7f28e5c00700 1 -- 172.21.15.37:0/3633207307 wait complete. 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.611+0000 7f28e5c00700 1 Processor -- start 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.611+0000 7f28e5c00700 1 -- start start 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.611+0000 7f28e5c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006afe0 0x7f28e019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.611+0000 7f28e5c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006b900 0x7f28e019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28e5c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e010f4e0 0x7f28e019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28e5c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0114e60 con 0x7f28e006b900 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28e5c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0114ce0 con 0x7f28e010f4e0 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28e5c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f28e0114fe0 con 0x7f28e006afe0 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006afe0 0x7f28e019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28de000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006b900 0x7f28e019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006afe0 0x7f28e019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35712/0 (socket says 172.21.15.37:35712) 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 -- 172.21.15.37:0/1588751641 learned_addr learned my addr 172.21.15.37:0/1588751641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28df400700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e010f4e0 0x7f28e019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:55.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 -- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e010f4e0 msgr2=0x7f28e019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f28e010f4e0 0x7f28e019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 -- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006b900 msgr2=0x7f28e019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006b900 0x7f28e019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28dea00700 1 -- 172.21.15.37:0/1588751641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e01a03c0 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.612+0000 7f28de000700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28e006b900 0x7f28e019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.613+0000 7f28dea00700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006afe0 0x7f28e019b3b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f28d0000c00 tx=0x7f28d00057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.613+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0027070 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.613+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d0022470 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.613+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e01a0650 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.613+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0017650 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.613+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e01a7f30 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.614+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f28e0063150 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.618+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28d002a070 con 0x7f28e006afe0 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.619+0000 7f28d7400700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f28c40a3d50 0x7f28c40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:55.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.619+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f28d00d1340 con 0x7f28e006afe0 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.619+0000 7f28de000700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f28c40a3d50 0x7f28c40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.620+0000 7f28d7400700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] conn(0x7f28c40ad760 0x7f28c40afc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.620+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 --> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28d0001d30 con 0x7f28c40ad760 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.620+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f28d00d16f0 con 0x7f28e006afe0 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.620+0000 7f28df400700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] conn(0x7f28c40ad760 0x7f28c40afc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.620+0000 7f28de000700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f28c40a3d50 0x7f28c40a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28e0102ef0 tx=0x7f28cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:55.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.621+0000 7f28df400700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] conn(0x7f28c40ad760 0x7f28c40afc00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.622+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== osd.4 v2:172.21.15.45:6808/2655399614 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28d0001d30 con 0x7f28c40ad760 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.641+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 --> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f28e0063580 con 0x7f28c40ad760 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28d7400700 1 -- 172.21.15.37:0/1588751641 <== osd.4 v2:172.21.15.45:6808/2655399614 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f28e0063580 con 0x7f28c40ad760 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] conn(0x7f28c40ad760 msgr2=0x7f28c40afc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28e5c00700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] conn(0x7f28c40ad760 0x7f28c40afc00 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f28c40a3d50 msgr2=0x7f28c40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28e5c00700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f28c40a3d50 0x7f28c40a6210 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f28e0102ef0 tx=0x7f28cc009260 comp rx=0 tx=0).stop 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006afe0 msgr2=0x7f28e019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28e5c00700 1 --2- 172.21.15.37:0/1588751641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f28e006afe0 0x7f28e019b3b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f28d0000c00 tx=0x7f28d00057c0 comp rx=0 tx=0).stop 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.642+0000 7f28df400700 1 -- 172.21.15.37:0/1588751641 reap_dead start 2024-04-26T01:55:55.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.643+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 shutdown_connections 2024-04-26T01:55:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.643+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 >> 172.21.15.37:0/1588751641 conn(0x7f28e00788b0 msgr2=0x7f28e0110a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.643+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 shutdown_connections 2024-04-26T01:55:55.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:55.643+0000 7f28e5c00700 1 -- 172.21.15.37:0/1588751641 wait complete. 2024-04-26T01:55:56.282 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:56 smithi037 ceph-mon[27906]: pgmap v387: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:56.288 INFO:teuthology.orchestra.run.smithi037.stdout:201863462978 2024-04-26T01:55:56.288 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T01:55:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:56 smithi071 ceph-mon[32578]: pgmap v387: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:56 smithi045 ceph-mon[32520]: pgmap v387: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:57.133 INFO:teuthology.orchestra.run.smithi037.stdout:167503724617 2024-04-26T01:55:57.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 -- 172.21.15.37:0/3411405558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b9c103b30 msgr2=0x7f6b9c103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 --2- 172.21.15.37:0/3411405558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b9c103b30 0x7f6b9c103f10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c009990 tx=0x7f6b8c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 -- 172.21.15.37:0/3411405558 shutdown_connections 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 --2- 172.21.15.37:0/3411405558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b9c103b30 0x7f6b9c103f10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 --2- 172.21.15.37:0/3411405558 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b9c10cf60 0x7f6b9c10f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 --2- 172.21.15.37:0/3411405558 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c104450 0x7f6b9c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.906+0000 7f6b9b200700 1 -- 172.21.15.37:0/3411405558 >> 172.21.15.37:0/3411405558 conn(0x7f6b9c0fd360 msgr2=0x7f6b9c0ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.907+0000 7f6b9b200700 1 -- 172.21.15.37:0/3411405558 shutdown_connections 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.907+0000 7f6b9b200700 1 -- 172.21.15.37:0/3411405558 wait complete. 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.908+0000 7f6b9b200700 1 Processor -- start 2024-04-26T01:55:57.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.908+0000 7f6b9b200700 1 -- start start 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b9b200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103b30 0x7f6b9c19b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b9b200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b9c104450 0x7f6b9c19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b9b200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b9c10cf60 0x7f6b9c19fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b9b200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c112ba0 con 0x7f6b9c10cf60 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b9b200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c112a20 con 0x7f6b9c104450 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b9b200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6b9c112d20 con 0x7f6b9c103b30 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103b30 0x7f6b9c19b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103b30 0x7f6b9c19b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35730/0 (socket says 172.21.15.37:35730) 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 -- 172.21.15.37:0/212190619 learned_addr learned my addr 172.21.15.37:0/212190619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99400700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b9c104450 0x7f6b9c19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 -- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b9c104450 msgr2=0x7f6b9c19b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6b9c104450 0x7f6b9c19b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 -- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b9c10cf60 msgr2=0x7f6b9c19fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:57.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b9c10cf60 0x7f6b9c19fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.909+0000 7f6b99e00700 1 -- 172.21.15.37:0/212190619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b9c1a0300 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.910+0000 7f6b99e00700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103b30 0x7f6b9c19b2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005910 tx=0x7f6b8c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.910+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c027070 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.910+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b9c1a0590 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.910+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b8c003f20 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.910+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c022d00 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.910+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b9c1a7e70 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.911+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6b9c103210 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.913+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6b8c02a070 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.915+0000 7f6b92a00700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6b800a3cd0 0x7f6b800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.915+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f6b8c0d14d0 con 0x7f6b9c103b30 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.915+0000 7f6b99400700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6b800a3cd0 0x7f6b800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.915+0000 7f6b92a00700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] conn(0x7f6b800ad810 0x7f6b800afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:57.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.915+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 --> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b8c0043d0 con 0x7f6b800ad810 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.915+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6b8c0d8050 con 0x7f6b9c103b30 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.916+0000 7f6b9a800700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] conn(0x7f6b800ad810 0x7f6b800afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.916+0000 7f6b99400700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6b800a3cd0 0x7f6b800a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c10e850 tx=0x7f6b8800b360 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.916+0000 7f6b9a800700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] conn(0x7f6b800ad810 0x7f6b800afcb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.917+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== osd.10 v2:172.21.15.45:6824/685235044 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6b8c0043d0 con 0x7f6b800ad810 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.936+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 --> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6b9c06b000 con 0x7f6b800ad810 2024-04-26T01:55:57.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.936+0000 7f6b92a00700 1 -- 172.21.15.37:0/212190619 <== osd.10 v2:172.21.15.45:6824/685235044 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6b9c06b000 con 0x7f6b800ad810 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] conn(0x7f6b800ad810 msgr2=0x7f6b800afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9b200700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] conn(0x7f6b800ad810 0x7f6b800afcb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6b800a3cd0 msgr2=0x7f6b800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9b200700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f6b800a3cd0 0x7f6b800a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c10e850 tx=0x7f6b8800b360 comp rx=0 tx=0).stop 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103b30 msgr2=0x7f6b9c19b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9b200700 1 --2- 172.21.15.37:0/212190619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6b9c103b30 0x7f6b9c19b2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c005910 tx=0x7f6b8c0047b0 comp rx=0 tx=0).stop 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.937+0000 7f6b9a800700 1 -- 172.21.15.37:0/212190619 reap_dead start 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.938+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 shutdown_connections 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.938+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 >> 172.21.15.37:0/212190619 conn(0x7f6b9c0fd360 msgr2=0x7f6b9c0ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.938+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 shutdown_connections 2024-04-26T01:55:57.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:56.938+0000 7f6b9b200700 1 -- 172.21.15.37:0/212190619 wait complete. 2024-04-26T01:55:57.976 INFO:teuthology.orchestra.run.smithi037.stdout:146028888143 2024-04-26T01:55:57.976 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T01:55:58.092 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:55:58 smithi037 ceph-mon[27906]: pgmap v388: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:58.299 INFO:teuthology.orchestra.run.smithi037.stdout:313532612653 2024-04-26T01:55:58.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T01:55:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:55:58 smithi071 ceph-mon[32578]: pgmap v388: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:58.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:55:58 smithi045 ceph-mon[32520]: pgmap v388: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:55:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 -- 172.21.15.37:0/177410873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c06b0f0 msgr2=0x7f4a0c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 --2- 172.21.15.37:0/177410873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c06b0f0 0x7f4a0c10f900 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f4a0400b0a0 tx=0x7f4a0400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 -- 172.21.15.37:0/177410873 shutdown_connections 2024-04-26T01:55:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 --2- 172.21.15.37:0/177410873 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c06b0f0 0x7f4a0c10f900 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 --2- 172.21.15.37:0/177410873 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a0c108a30 0x7f4a0c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 --2- 172.21.15.37:0/177410873 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c06a750 0x7f4a0c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 -- 172.21.15.37:0/177410873 >> 172.21.15.37:0/177410873 conn(0x7f4a0c0788f0 msgr2=0x7f4a0c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 -- 172.21.15.37:0/177410873 shutdown_connections 2024-04-26T01:55:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.533+0000 7f4a13a00700 1 -- 172.21.15.37:0/177410873 wait complete. 2024-04-26T01:55:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.534+0000 7f4a13a00700 1 Processor -- start 2024-04-26T01:55:58.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.534+0000 7f4a13a00700 1 -- start start 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a13a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a0c06a750 0x7f4a0c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a13a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c06b0f0 0x7f4a0c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a13a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c108a30 0x7f4a0c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a13a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c112c60 con 0x7f4a0c108a30 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a13a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c112ae0 con 0x7f4a0c06a750 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a13a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0c112de0 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a11000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a0c06a750 0x7f4a0c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a11a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c108a30 0x7f4a0c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a0be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c06b0f0 0x7f4a0c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a11000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a0c06a750 0x7f4a0c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:51900/0 (socket says 172.21.15.37:51900) 2024-04-26T01:55:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a11a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c108a30 0x7f4a0c19da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52310/0 (socket says 172.21.15.37:52310) 2024-04-26T01:55:58.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.535+0000 7f4a11000700 1 -- 172.21.15.37:0/66997059 learned_addr learned my addr 172.21.15.37:0/66997059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a0be00700 1 -- 172.21.15.37:0/66997059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a0c06a750 msgr2=0x7f4a0c199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a0be00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4a0c06a750 0x7f4a0c199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a0be00700 1 -- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c108a30 msgr2=0x7f4a0c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a0be00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a0c108a30 0x7f4a0c19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a0be00700 1 -- 172.21.15.37:0/66997059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0c19e160 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a0be00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c06b0f0 0x7f4a0c199690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49f800e9f0 tx=0x7f49f800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8004220 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a0c19e450 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49f80043c0 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f8005020 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.536+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a0c1a5cd0 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.537+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a0c0631d0 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.539+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f49f8004750 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.540+0000 7f4a09600700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f49f40a3cd0 0x7f49f40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.541+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f49f80d07b0 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.541+0000 7f4a11000700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f49f40a3cd0 0x7f49f40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.541+0000 7f4a09600700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] conn(0x7f49f40ad810 0x7f49f40afcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:55:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.541+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 --> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49f8016ec0 con 0x7f49f40ad810 2024-04-26T01:55:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.541+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49f80d0c00 con 0x7f4a0c06b0f0 2024-04-26T01:55:58.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.541+0000 7f4a11a00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] conn(0x7f49f40ad810 0x7f49f40afcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:55:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.542+0000 7f4a11000700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f49f40a3cd0 0x7f49f40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009680 tx=0x7f4a00009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.542+0000 7f4a11a00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] conn(0x7f49f40ad810 0x7f49f40afcb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:55:58.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.543+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== osd.9 v2:172.21.15.37:6826/759844369 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f49f8016ec0 con 0x7f49f40ad810 2024-04-26T01:55:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.562+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 --> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a0c063650 con 0x7f49f40ad810 2024-04-26T01:55:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.562+0000 7f4a09600700 1 -- 172.21.15.37:0/66997059 <== osd.9 v2:172.21.15.37:6826/759844369 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4a0c063650 con 0x7f49f40ad810 2024-04-26T01:55:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] conn(0x7f49f40ad810 msgr2=0x7f49f40afcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] conn(0x7f49f40ad810 0x7f49f40afcb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:55:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f49f40a3cd0 msgr2=0x7f49f40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f49f40a3cd0 0x7f49f40a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009680 tx=0x7f4a00009260 comp rx=0 tx=0).stop 2024-04-26T01:55:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c06b0f0 msgr2=0x7f4a0c199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:55:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 --2- 172.21.15.37:0/66997059 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4a0c06b0f0 0x7f4a0c199690 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49f800e9f0 tx=0x7f49f800ed00 comp rx=0 tx=0).stop 2024-04-26T01:55:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a11a00700 1 -- 172.21.15.37:0/66997059 reap_dead start 2024-04-26T01:55:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 shutdown_connections 2024-04-26T01:55:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 >> 172.21.15.37:0/66997059 conn(0x7f4a0c0788f0 msgr2=0x7f4a0c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:55:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 shutdown_connections 2024-04-26T01:55:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:55:58.563+0000 7f4a13a00700 1 -- 172.21.15.37:0/66997059 wait complete. 2024-04-26T01:55:58.915 INFO:teuthology.orchestra.run.smithi037.stdout:34359738467 2024-04-26T01:55:58.916 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T01:55:59.155 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:59.778 INFO:teuthology.orchestra.run.smithi037.stdout:292057776179 2024-04-26T01:55:59.778 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T01:55:59.793 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:59.800 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:59.813 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:59.823 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:59.826 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:55:59.837 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:00.070 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:00.304 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:00 smithi037 ceph-mon[27906]: pgmap v389: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:00 smithi071 ceph-mon[32578]: pgmap v389: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:00 smithi045 ceph-mon[32520]: pgmap v389: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:02 smithi037 ceph-mon[27906]: pgmap v390: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:56:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:02 smithi071 ceph-mon[32578]: pgmap v390: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:56:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:02 smithi045 ceph-mon[32520]: pgmap v390: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T01:56:02.868 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:03.807 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:04.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:04 smithi037 ceph-mon[27906]: pgmap v391: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:04 smithi071 ceph-mon[32578]: pgmap v391: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:04 smithi045 ceph-mon[32520]: pgmap v391: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:05.344 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:05 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:05 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:05.499 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:05 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:56:06.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:06 smithi037 ceph-mon[27906]: pgmap v392: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:06 smithi071 ceph-mon[32578]: pgmap v392: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:06 smithi045 ceph-mon[32520]: pgmap v392: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:06.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 -- 172.21.15.37:0/3092634108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 msgr2=0x7f0794111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 --2- 172.21.15.37:0/3092634108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f0794111830 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f078c009f90 tx=0x7f078c00b330 comp rx=0 tx=0).stop 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 -- 172.21.15.37:0/3092634108 shutdown_connections 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 --2- 172.21.15.37:0/3092634108 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f0794111830 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 --2- 172.21.15.37:0/3092634108 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794074c70 0x7f0794075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 --2- 172.21.15.37:0/3092634108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0794075590 0x7f079410ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 -- 172.21.15.37:0/3092634108 >> 172.21.15.37:0/3092634108 conn(0x7f0794079ba0 msgr2=0x7f0794079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 -- 172.21.15.37:0/3092634108 shutdown_connections 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.804+0000 7f0798c00700 1 -- 172.21.15.37:0/3092634108 wait complete. 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.805+0000 7f0798c00700 1 Processor -- start 2024-04-26T01:56:06.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.805+0000 7f0798c00700 1 -- start start 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0798c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0794074c70 0x7f0794102b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0798c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 0x7f07941030a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0798c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f07941035e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0798c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0794114e40 con 0x7f0794074c70 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0798c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0794114cc0 con 0x7f079410f440 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0798c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0794114fc0 con 0x7f0794075590 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0791e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0794074c70 0x7f0794102b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0791400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 0x7f07941030a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0791400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 0x7f07941030a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44560/0 (socket says 172.21.15.37:44560) 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0791400700 1 -- 172.21.15.37:0/3408200136 learned_addr learned my addr 172.21.15.37:0/3408200136 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.806+0000 7f0792800700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f07941035e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0791400700 1 -- 172.21.15.37:0/3408200136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 msgr2=0x7f07941035e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0791400700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f07941035e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0791400700 1 -- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0794074c70 msgr2=0x7f0794102b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0791400700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0794074c70 0x7f0794102b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0791400700 1 -- 172.21.15.37:0/3408200136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07941ab9e0 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0792800700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f07941035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0791400700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 0x7f07941030a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f078000e9a0 tx=0x7f078000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780004220 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07800043c0 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07941abcd0 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0780003c70 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.807+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07941ac200 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.809+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0794076360 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.810+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f078000c7c0 con 0x7f0794075590 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.812+0000 7f078aa00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07780a3d50 0x7f07780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:06.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.812+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f07800d06b0 con 0x7f0794075590 2024-04-26T01:56:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.813+0000 7f0791e00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07780a3d50 0x7f07780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.813+0000 7f0791e00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07780a3d50 0x7f07780a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0784005fd0 tx=0x7f0784009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:06.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:06.816+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f078009cee0 con 0x7f0794075590 2024-04-26T01:56:07.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.108+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f07941ac490 con 0x7f0794075590 2024-04-26T01:56:07.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.108+0000 7f078aa00700 1 -- 172.21.15.37:0/3408200136 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f078009c7e0 con 0x7f0794075590 2024-04-26T01:56:07.108 INFO:teuthology.orchestra.run.smithi037.stdout:223338299456 2024-04-26T01:56:07.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.111+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07780a3d50 msgr2=0x7f07780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:07.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.111+0000 7f0798c00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07780a3d50 0x7f07780a6210 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0784005fd0 tx=0x7f0784009310 comp rx=0 tx=0).stop 2024-04-26T01:56:07.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.111+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 msgr2=0x7f07941030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:07.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.111+0000 7f0798c00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 0x7f07941030a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f078000e9a0 tx=0x7f078000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:56:07.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 shutdown_connections 2024-04-26T01:56:07.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0794074c70 0x7f0794102b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:07.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f07780a3d50 0x7f07780a6210 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:07.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f079410f440 0x7f07941035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:07.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 --2- 172.21.15.37:0/3408200136 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0794075590 0x7f07941030a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:07.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 >> 172.21.15.37:0/3408200136 conn(0x7f0794079ba0 msgr2=0x7f0794100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:07.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 shutdown_connections 2024-04-26T01:56:07.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:07.112+0000 7f0798c00700 1 -- 172.21.15.37:0/3408200136 wait complete. 2024-04-26T01:56:07.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.587 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: pgmap v393: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:07.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.588 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:07 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3408200136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:56:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: pgmap v393: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:07 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3408200136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: pgmap v393: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:07.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:07 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3408200136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T01:56:08.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299453 got 223338299456 for osd.7 2024-04-26T01:56:08.973 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:09.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:09 smithi071 ceph-mon[32578]: pgmap v394: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:09 smithi037 ceph-mon[27906]: pgmap v394: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:09 smithi045 ceph-mon[32520]: pgmap v394: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:11.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:11 smithi071 ceph-mon[32578]: pgmap v395: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:11.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:11 smithi037 ceph-mon[27906]: pgmap v395: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:12.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:11 smithi045 ceph-mon[32520]: pgmap v395: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.739+0000 7f9486e00700 1 -- 172.21.15.37:0/3178730210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006afe0 msgr2=0x7f948006b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.739+0000 7f9486e00700 1 --2- 172.21.15.37:0/3178730210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006afe0 0x7f948006b440 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f946c009990 tx=0x7f946c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 -- 172.21.15.37:0/3178730210 shutdown_connections 2024-04-26T01:56:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 --2- 172.21.15.37:0/3178730210 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006b980 0x7f948010f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 --2- 172.21.15.37:0/3178730210 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9480108a20 0x7f9480108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 --2- 172.21.15.37:0/3178730210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006afe0 0x7f948006b440 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 -- 172.21.15.37:0/3178730210 >> 172.21.15.37:0/3178730210 conn(0x7f94800788b0 msgr2=0x7f9480078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 -- 172.21.15.37:0/3178730210 shutdown_connections 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.740+0000 7f9486e00700 1 -- 172.21.15.37:0/3178730210 wait complete. 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.741+0000 7f9486e00700 1 Processor -- start 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.741+0000 7f9486e00700 1 -- start start 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.741+0000 7f9486e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 0x7f948019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f9486e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006b980 0x7f948019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f9486e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9480108a20 0x7f948019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f9486e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9480112c10 con 0x7f948006b980 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f9486e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9480112a90 con 0x7f948006afe0 2024-04-26T01:56:13.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f9486e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9480112d90 con 0x7f9480108a20 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 0x7f948019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006b980 0x7f948019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 0x7f948019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:57066/0 (socket says 172.21.15.37:57066) 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 -- 172.21.15.37:0/2023106574 learned_addr learned my addr 172.21.15.37:0/2023106574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f9484e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9480108a20 0x7f948019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 -- 172.21.15.37:0/2023106574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9480108a20 msgr2=0x7f948019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9480108a20 0x7f948019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 -- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006b980 msgr2=0x7f948019b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006b980 0x7f948019b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947fe00700 1 -- 172.21.15.37:0/2023106574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94801a0390 con 0x7f948006afe0 2024-04-26T01:56:13.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.742+0000 7f947f400700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006b980 0x7f948019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.743+0000 7f947fe00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 0x7f948019b380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f947000e9a0 tx=0x7f947000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.743+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947000ca10 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.743+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94801a0680 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.743+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947000cbb0 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.743+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94801a7f00 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.743+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470018420 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.744+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9460005330 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.747+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9470010430 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.748+0000 7f947cc00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f94640a3cd0 0x7f94640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.748+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f94700d0640 con 0x7f948006afe0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.748+0000 7f947f400700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f94640a3cd0 0x7f94640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.749+0000 7f947f400700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f94640a3cd0 0x7f94640a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9480078f00 tx=0x7f946c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.752+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f947009ce70 con 0x7f948006afe0 2024-04-26T01:56:13.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.993+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f94600058f0 con 0x7f948006afe0 2024-04-26T01:56:13.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.994+0000 7f947cc00700 1 -- 172.21.15.37:0/2023106574 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f947009c770 con 0x7f948006afe0 2024-04-26T01:56:13.020 INFO:teuthology.orchestra.run.smithi037.stdout:257698037820 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f94640a3cd0 msgr2=0x7f94640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f94640a3cd0 0x7f94640a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9480078f00 tx=0x7f946c00b260 comp rx=0 tx=0).stop 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 msgr2=0x7f948019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 0x7f948019b380 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f947000e9a0 tx=0x7f947000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 shutdown_connections 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f94640a3cd0 0x7f94640a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9480108a20 0x7f948019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f948006afe0 0x7f948019b380 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 --2- 172.21.15.37:0/2023106574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f948006b980 0x7f948019b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 >> 172.21.15.37:0/2023106574 conn(0x7f94800788b0 msgr2=0x7f948010df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 shutdown_connections 2024-04-26T01:56:13.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:12.997+0000 7f9486e00700 1 -- 172.21.15.37:0/2023106574 wait complete. 2024-04-26T01:56:13.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:13 smithi037 ceph-mon[27906]: pgmap v396: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:13.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:13 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2023106574' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:56:13.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 -- 172.21.15.37:0/2172340497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4108a50 msgr2=0x7f44f4108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 --2- 172.21.15.37:0/2172340497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4108a50 0x7f44f4108e30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f44e4009990 tx=0x7f44e4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 -- 172.21.15.37:0/2172340497 shutdown_connections 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 --2- 172.21.15.37:0/2172340497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4108a50 0x7f44f4108e30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 --2- 172.21.15.37:0/2172340497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 --2- 172.21.15.37:0/2172340497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f4102a90 0x7f44f4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 -- 172.21.15.37:0/2172340497 >> 172.21.15.37:0/2172340497 conn(0x7f44f4079ba0 msgr2=0x7f44f4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 -- 172.21.15.37:0/2172340497 shutdown_connections 2024-04-26T01:56:13.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.821+0000 7f44fb200700 1 -- 172.21.15.37:0/2172340497 wait complete. 2024-04-26T01:56:13.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.822+0000 7f44fb200700 1 Processor -- start 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 -- start start 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4102a90 0x7f44f419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f419bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f419fe80 0x7f44f41a0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4112c60 con 0x7f44f4102a90 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4112ae0 con 0x7f44f419fe80 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44fb200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f44f4112de0 con 0x7f44f4103450 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44ebe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f419bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44f3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4102a90 0x7f44f419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44ebe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f419bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47824/0 (socket says 172.21.15.37:47824) 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44ebe00700 1 -- 172.21.15.37:0/2283459007 learned_addr learned my addr 172.21.15.37:0/2283459007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:13.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.823+0000 7f44f9200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f419fe80 0x7f44f41a0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f3e00700 1 -- 172.21.15.37:0/2283459007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 msgr2=0x7f44f419bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f3e00700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f419bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f3e00700 1 -- 172.21.15.37:0/2283459007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f419fe80 msgr2=0x7f44f41a0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:13.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f3e00700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f419fe80 0x7f44f41a0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f3e00700 1 -- 172.21.15.37:0/2283459007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f41a09a0 con 0x7f44f4102a90 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f9200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f419fe80 0x7f44f41a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44ebe00700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.824+0000 7f44f3e00700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4102a90 0x7f44f419b650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44e4008000 tx=0x7f44e40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.825+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4026070 con 0x7f44f4102a90 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.825+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e4021410 con 0x7f44f4102a90 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.825+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f41a0c30 con 0x7f44f4102a90 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.825+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e4017650 con 0x7f44f4102a90 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.825+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f41a83b0 con 0x7f44f4102a90 2024-04-26T01:56:13.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.826+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f4069730 con 0x7f44f4102a90 2024-04-26T01:56:13.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.828+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f44e4029070 con 0x7f44f4102a90 2024-04-26T01:56:13.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.830+0000 7f44f1600700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f44d80a3cc0 0x7f44d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:13.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.830+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f44e40d0e00 con 0x7f44f4102a90 2024-04-26T01:56:13.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.830+0000 7f44ebe00700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f44d80a3cc0 0x7f44d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:13.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.831+0000 7f44ebe00700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f44d80a3cc0 0x7f44d80a6180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44f410fcf0 tx=0x7f44dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:13.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:13.833+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44e4030080 con 0x7f44f4102a90 2024-04-26T01:56:14.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:13 smithi045 ceph-mon[32520]: pgmap v396: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:14.023 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:13 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2023106574' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:56:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.069+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f44f41a85e0 con 0x7f44f4102a90 2024-04-26T01:56:14.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.070+0000 7f44f1600700 1 -- 172.21.15.37:0/2283459007 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f44e4013070 con 0x7f44f4102a90 2024-04-26T01:56:14.070 INFO:teuthology.orchestra.run.smithi037.stdout:111669149784 2024-04-26T01:56:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f44d80a3cc0 msgr2=0x7f44d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f44d80a3cc0 0x7f44d80a6180 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f44f410fcf0 tx=0x7f44dc009260 comp rx=0 tx=0).stop 2024-04-26T01:56:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4102a90 msgr2=0x7f44f419b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:14.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4102a90 0x7f44f419b650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f44e4008000 tx=0x7f44e40057c0 comp rx=0 tx=0).stop 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 shutdown_connections 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44f4102a90 0x7f44f419b650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f44d80a3cc0 0x7f44d80a6180 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f44f419fe80 0x7f44f41a0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 --2- 172.21.15.37:0/2283459007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f44f4103450 0x7f44f419bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 >> 172.21.15.37:0/2283459007 conn(0x7f44f4079ba0 msgr2=0x7f44f4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 shutdown_connections 2024-04-26T01:56:14.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:14.074+0000 7f44fb200700 1 -- 172.21.15.37:0/2283459007 wait complete. 2024-04-26T01:56:14.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:13 smithi071 ceph-mon[32578]: pgmap v396: 97 pgs: 97 active+clean; 638 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:14.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:13 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2023106574' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T01:56:14.441 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:14.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:14 smithi037 ceph-mon[27906]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T01:56:14.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:14 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2283459007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:56:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:14 smithi045 ceph-mon[32520]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T01:56:15.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:14 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2283459007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:56:15.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:14 smithi071 ceph-mon[32578]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-04-26T01:56:15.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:14 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2283459007' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T01:56:15.912 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:15 smithi037 ceph-mon[27906]: pgmap v397: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:56:16.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:15 smithi045 ceph-mon[32520]: pgmap v397: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:56:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 -- 172.21.15.37:0/655746177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 msgr2=0x7f149c10ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 --2- 172.21.15.37:0/655746177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c10ef00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1488009990 tx=0x7f1488009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 -- 172.21.15.37:0/655746177 shutdown_connections 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 --2- 172.21.15.37:0/655746177 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 0x7f149c111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 --2- 172.21.15.37:0/655746177 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f149c074c70 0x7f149c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 --2- 172.21.15.37:0/655746177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c10ef00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 -- 172.21.15.37:0/655746177 >> 172.21.15.37:0/655746177 conn(0x7f149c079ba0 msgr2=0x7f149c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 -- 172.21.15.37:0/655746177 shutdown_connections 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.089+0000 7f14a2400700 1 -- 172.21.15.37:0/655746177 wait complete. 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.090+0000 7f14a2400700 1 Processor -- start 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.090+0000 7f14a2400700 1 -- start start 2024-04-26T01:56:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.090+0000 7f14a2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f149c074c70 0x7f149c19b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f14a2400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f14a2400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 0x7f149c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f14a2400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f149c114e50 con 0x7f149c075590 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f14a2400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f149c114cd0 con 0x7f149c10f440 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f14a2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f149c114fd0 con 0x7f149c074c70 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c19b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51994/0 (socket says 172.21.15.37:51994) 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 -- 172.21.15.37:0/179165263 learned_addr learned my addr 172.21.15.37:0/179165263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149be00700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 0x7f149c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149b400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f149c074c70 0x7f149c19b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 -- 172.21.15.37:0/179165263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f149c074c70 msgr2=0x7f149c19b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f149c074c70 0x7f149c19b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 -- 172.21.15.37:0/179165263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 msgr2=0x7f149c19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 0x7f149c19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149aa00700 1 -- 172.21.15.37:0/179165263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c1a0380 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.091+0000 7f149be00700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 0x7f149c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.092+0000 7f149aa00700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c19b900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f14880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.092+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488026070 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.092+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1488021410 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.092+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149c1a0610 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.092+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1488017650 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.092+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f149c1a7ef0 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.093+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f149c069730 con 0x7f149c075590 2024-04-26T01:56:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.096+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1488029070 con 0x7f149c075590 2024-04-26T01:56:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.097+0000 7f1493e00700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f14800a3cc0 0x7f14800a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.097+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f14880d0f80 con 0x7f149c075590 2024-04-26T01:56:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.098+0000 7f149b400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f14800a3cc0 0x7f14800a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.098+0000 7f149b400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f14800a3cc0 0x7f14800a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f148c009c60 tx=0x7f148c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.102+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1488013070 con 0x7f149c075590 2024-04-26T01:56:16.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:15 smithi071 ceph-mon[32578]: pgmap v397: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:56:16.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.340+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f149c02df80 con 0x7f149c075590 2024-04-26T01:56:16.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.341+0000 7f1493e00700 1 -- 172.21.15.37:0/179165263 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f148809d9a0 con 0x7f149c075590 2024-04-26T01:56:16.340 INFO:teuthology.orchestra.run.smithi037.stdout:55834574948 2024-04-26T01:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f14800a3cc0 msgr2=0x7f14800a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f14800a3cc0 0x7f14800a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f148c009c60 tx=0x7f148c009260 comp rx=0 tx=0).stop 2024-04-26T01:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 msgr2=0x7f149c19b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:16.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c19b900 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f1488000c00 tx=0x7f14880047b0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 shutdown_connections 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f14800a3cc0 0x7f14800a6180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f149c10f440 0x7f149c19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f149c074c70 0x7f149c19b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 --2- 172.21.15.37:0/179165263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f149c075590 0x7f149c19b900 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 >> 172.21.15.37:0/179165263 conn(0x7f149c079ba0 msgr2=0x7f149c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 shutdown_connections 2024-04-26T01:56:16.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.344+0000 7f14a2400700 1 -- 172.21.15.37:0/179165263 wait complete. 2024-04-26T01:56:16.934 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:16 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/179165263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:56:17.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:16 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/179165263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:56:17.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:16 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/179165263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T01:56:17.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:17 smithi037 ceph-mon[27906]: pgmap v398: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:56:17.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:17 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/379461999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:56:18.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:17 smithi045 ceph-mon[32520]: pgmap v398: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:56:18.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:17 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/379461999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:56:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:17 smithi071 ceph-mon[32578]: pgmap v398: 97 pgs: 97 active+clean; 638 KiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 0 B/s wr, 1 op/s 2024-04-26T01:56:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:17 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/379461999' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T01:56:18.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 -- 172.21.15.37:0/1996379426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10e3b0 msgr2=0x7f1d4c115f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:18.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 --2- 172.21.15.37:0/1996379426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10e3b0 0x7f1d4c115f80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1d440098f0 tx=0x7f1d44009c00 comp rx=0 tx=0).stop 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 -- 172.21.15.37:0/1996379426 shutdown_connections 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 --2- 172.21.15.37:0/1996379426 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c10e3b0 0x7f1d4c115f80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 --2- 172.21.15.37:0/1996379426 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c1096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 --2- 172.21.15.37:0/1996379426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c109c80 0x7f1d4c10dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 -- 172.21.15.37:0/1996379426 >> 172.21.15.37:0/1996379426 conn(0x7f1d4c076210 msgr2=0x7f1d4c107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 -- 172.21.15.37:0/1996379426 shutdown_connections 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.333+0000 7f1d52200700 1 -- 172.21.15.37:0/1996379426 wait complete. 2024-04-26T01:56:18.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.334+0000 7f1d52200700 1 Processor -- start 2024-04-26T01:56:18.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 -- start start 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c109c80 0x7f1d4c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d4c10e3b0 0x7f1d4c1a3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c1192a0 con 0x7f1d4c1092d0 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c119120 con 0x7f1d4c109c80 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.335+0000 7f1d52200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4c119420 con 0x7f1d4c10e3b0 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52014/0 (socket says 172.21.15.37:52014) 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 -- 172.21.15.37:0/3925151262 learned_addr learned my addr 172.21.15.37:0/3925151262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d42a00700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c109c80 0x7f1d4c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:18.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4be00700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d4c10e3b0 0x7f1d4c1a3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 -- 172.21.15.37:0/3925151262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d4c10e3b0 msgr2=0x7f1d4c1a3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d4c10e3b0 0x7f1d4c1a3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 -- 172.21.15.37:0/3925151262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c109c80 msgr2=0x7f1d4c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c109c80 0x7f1d4c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 -- 172.21.15.37:0/3925151262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d4c1a4720 con 0x7f1d4c1092d0 2024-04-26T01:56:18.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.336+0000 7f1d4b400700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c19f710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00ef10 tx=0x7f1d3c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.337+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c00c9c0 con 0x7f1d4c1092d0 2024-04-26T01:56:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.337+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d3c019070 con 0x7f1d4c1092d0 2024-04-26T01:56:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.337+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d4c1a49b0 con 0x7f1d4c1092d0 2024-04-26T01:56:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.337+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c014480 con 0x7f1d4c1092d0 2024-04-26T01:56:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.337+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d4c1ac290 con 0x7f1d4c1092d0 2024-04-26T01:56:18.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.338+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d4c069730 con 0x7f1d4c1092d0 2024-04-26T01:56:18.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.340+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1d3c012050 con 0x7f1d4c1092d0 2024-04-26T01:56:18.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.341+0000 7f1d48c00700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1d2c0a3cc0 0x7f1d2c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:18.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.342+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f1d3c0d0300 con 0x7f1d4c1092d0 2024-04-26T01:56:18.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.342+0000 7f1d42a00700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1d2c0a3cc0 0x7f1d2c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:18.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.343+0000 7f1d42a00700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1d2c0a3cc0 0x7f1d2c0a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009750 tx=0x7f1d34009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:18.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.346+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d3c010040 con 0x7f1d4c1092d0 2024-04-26T01:56:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.582+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f1d4c02df80 con 0x7f1d4c1092d0 2024-04-26T01:56:18.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.582+0000 7f1d48c00700 1 -- 172.21.15.37:0/3925151262 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d3c09cc90 con 0x7f1d4c1092d0 2024-04-26T01:56:18.582 INFO:teuthology.orchestra.run.smithi037.stdout:77309411424 2024-04-26T01:56:18.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.585+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1d2c0a3cc0 msgr2=0x7f1d2c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:18.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.585+0000 7f1d52200700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1d2c0a3cc0 0x7f1d2c0a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1d34009750 tx=0x7f1d34009310 comp rx=0 tx=0).stop 2024-04-26T01:56:18.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.585+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 msgr2=0x7f1d4c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:18.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.585+0000 7f1d52200700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c19f710 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c00ef10 tx=0x7f1d3c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.586+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 shutdown_connections 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.586+0000 7f1d52200700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d4c1092d0 0x7f1d4c19f710 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.586+0000 7f1d52200700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1d2c0a3cc0 0x7f1d2c0a6180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.586+0000 7f1d52200700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1d4c10e3b0 0x7f1d4c1a3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.586+0000 7f1d52200700 1 --2- 172.21.15.37:0/3925151262 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1d4c109c80 0x7f1d4c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.586+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 >> 172.21.15.37:0/3925151262 conn(0x7f1d4c076210 msgr2=0x7f1d4c076790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.587+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 shutdown_connections 2024-04-26T01:56:18.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:18.587+0000 7f1d52200700 1 -- 172.21.15.37:0/3925151262 wait complete. 2024-04-26T01:56:18.834 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:18 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3925151262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:56:19.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:18 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3925151262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:56:19.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:18 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3925151262' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T01:56:19.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:19 smithi037 ceph-mon[27906]: pgmap v399: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:19.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:19 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3575257884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:56:20.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:19 smithi045 ceph-mon[32520]: pgmap v399: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:20.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:19 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3575257884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 -- 172.21.15.37:0/1558460267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c06b0f0 msgr2=0x7fd08c10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 --2- 172.21.15.37:0/1558460267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c06b0f0 0x7fd08c10f900 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd08400b0a0 tx=0x7fd08400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 -- 172.21.15.37:0/1558460267 shutdown_connections 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 --2- 172.21.15.37:0/1558460267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c108a30 0x7fd08c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 --2- 172.21.15.37:0/1558460267 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c06b0f0 0x7fd08c10f900 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 --2- 172.21.15.37:0/1558460267 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06a750 0x7fd08c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 -- 172.21.15.37:0/1558460267 >> 172.21.15.37:0/1558460267 conn(0x7fd08c0788f0 msgr2=0x7fd08c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.971+0000 7fd091600700 1 -- 172.21.15.37:0/1558460267 shutdown_connections 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.972+0000 7fd091600700 1 -- 172.21.15.37:0/1558460267 wait complete. 2024-04-26T01:56:20.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.973+0000 7fd091600700 1 Processor -- start 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.973+0000 7fd091600700 1 -- start start 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd091600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 0x7fd08c101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd091600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06b0f0 0x7fd08c101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd091600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c108a30 0x7fd08c102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd091600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112c10 con 0x7fd08c06a750 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd091600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112a90 con 0x7fd08c108a30 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd091600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd08c112d90 con 0x7fd08c06b0f0 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd08aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 0x7fd08c101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd08aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 0x7fd08c101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52046/0 (socket says 172.21.15.37:52046) 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd08aa00700 1 -- 172.21.15.37:0/4097133684 learned_addr learned my addr 172.21.15.37:0/4097133684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd08a000700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06b0f0 0x7fd08c101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.974+0000 7fd08b400700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c108a30 0x7fd08c102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08aa00700 1 -- 172.21.15.37:0/4097133684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06b0f0 msgr2=0x7fd08c101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08aa00700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06b0f0 0x7fd08c101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08aa00700 1 -- 172.21.15.37:0/4097133684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c108a30 msgr2=0x7fd08c102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08aa00700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c108a30 0x7fd08c102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08aa00700 1 -- 172.21.15.37:0/4097133684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd08c1a76d0 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08a000700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06b0f0 0x7fd08c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08b400700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c108a30 0x7fd08c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd08aa00700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 0x7fd08c101a10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00eb30 tx=0x7fd07c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c00cbd0 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd07c0044d0 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd08c1a79c0 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.975+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c003e30 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.976+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08c1a7f00 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.977+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd08c069730 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.981+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd07c004000 con 0x7fd08c06a750 2024-04-26T01:56:20.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.983+0000 7fd083400700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0700a3c40 0x7fd0700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.983+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd07c0d0930 con 0x7fd08c06a750 2024-04-26T01:56:20.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.983+0000 7fd08a000700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0700a3c40 0x7fd0700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.984+0000 7fd08a000700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0700a3c40 0x7fd0700a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd08c078f60 tx=0x7fd078009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.985+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd07c053ff0 con 0x7fd08c06a750 2024-04-26T01:56:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:19 smithi071 ceph-mon[32578]: pgmap v399: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:19 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3575257884' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T01:56:20.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.222+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fd08c02df80 con 0x7fd08c06a750 2024-04-26T01:56:20.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.223+0000 7fd083400700 1 -- 172.21.15.37:0/4097133684 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fd07c0042f0 con 0x7fd08c06a750 2024-04-26T01:56:20.222 INFO:teuthology.orchestra.run.smithi037.stdout:347892351022 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.225+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0700a3c40 msgr2=0x7fd0700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.225+0000 7fd091600700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0700a3c40 0x7fd0700a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd08c078f60 tx=0x7fd078009260 comp rx=0 tx=0).stop 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.225+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 msgr2=0x7fd08c101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.225+0000 7fd091600700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 0x7fd08c101a10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd07c00eb30 tx=0x7fd07c00ee40 comp rx=0 tx=0).stop 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 shutdown_connections 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd08c06a750 0x7fd08c101a10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd0700a3c40 0x7fd0700a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd08c108a30 0x7fd08c102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 --2- 172.21.15.37:0/4097133684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd08c06b0f0 0x7fd08c101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 >> 172.21.15.37:0/4097133684 conn(0x7fd08c0788f0 msgr2=0x7fd08c0ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 shutdown_connections 2024-04-26T01:56:20.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.226+0000 7fd091600700 1 -- 172.21.15.37:0/4097133684 wait complete. 2024-04-26T01:56:20.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 -- 172.21.15.37:0/25710841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 msgr2=0x7f767010c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 --2- 172.21.15.37:0/25710841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f767010c940 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f765800b0a0 tx=0x7f765800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 -- 172.21.15.37:0/25710841 shutdown_connections 2024-04-26T01:56:20.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 --2- 172.21.15.37:0/25710841 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f767010cf60 0x7f767010f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 --2- 172.21.15.37:0/25710841 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7670103b30 0x7f7670103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 --2- 172.21.15.37:0/25710841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f767010c940 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.898+0000 7f766fe00700 1 -- 172.21.15.37:0/25710841 >> 172.21.15.37:0/25710841 conn(0x7f76700fd360 msgr2=0x7f76700ff780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:20.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.899+0000 7f766fe00700 1 -- 172.21.15.37:0/25710841 shutdown_connections 2024-04-26T01:56:20.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.899+0000 7f766fe00700 1 -- 172.21.15.37:0/25710841 wait complete. 2024-04-26T01:56:20.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 Processor -- start 2024-04-26T01:56:20.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 -- start start 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7670103b30 0x7f7670101790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f7670101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767010cf60 0x7f7670102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7670112b60 con 0x7f7670104450 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f76701129e0 con 0x7f767010cf60 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.900+0000 7f766fe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7670112ce0 con 0x7f7670103b30 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f7670101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f7670101cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35742/0 (socket says 172.21.15.37:35742) 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 -- 172.21.15.37:0/1116012175 learned_addr learned my addr 172.21.15.37:0/1116012175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f766ea00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7670103b30 0x7f7670101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f766f400700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767010cf60 0x7f7670102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 -- 172.21.15.37:0/1116012175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7670103b30 msgr2=0x7f7670101790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7670103b30 0x7f7670101790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 -- 172.21.15.37:0/1116012175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767010cf60 msgr2=0x7f7670102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:20.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767010cf60 0x7f7670102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:20.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.901+0000 7f7666000700 1 -- 172.21.15.37:0/1116012175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76701a75b0 con 0x7f7670104450 2024-04-26T01:56:20.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.902+0000 7f7666000700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f7670101cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f765800d010 tx=0x7f7658004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.902+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658021070 con 0x7f7670104450 2024-04-26T01:56:20.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.902+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7658007e30 con 0x7f7670104450 2024-04-26T01:56:20.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.902+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658004520 con 0x7f7670104450 2024-04-26T01:56:20.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.902+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76701a7840 con 0x7f7670104450 2024-04-26T01:56:20.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.902+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76701a7d30 con 0x7f7670104450 2024-04-26T01:56:20.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.903+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7670069730 con 0x7f7670104450 2024-04-26T01:56:20.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.905+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7658003c00 con 0x7f7670104450 2024-04-26T01:56:20.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.907+0000 7f7667e00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f76540a3c40 0x7f76540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:20.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.907+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f765800f020 con 0x7f7670104450 2024-04-26T01:56:20.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.907+0000 7f766ea00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f76540a3c40 0x7f76540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:20.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.908+0000 7f766ea00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f76540a3c40 0x7f76540a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7660009c60 tx=0x7f7660009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:20.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:20.911+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f765802a020 con 0x7f7670104450 2024-04-26T01:56:20.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:20 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4097133684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:56:21.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:20 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4097133684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 -- 172.21.15.37:0/1493610520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d8108a20 msgr2=0x7f31d8108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 --2- 172.21.15.37:0/1493610520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d8108a20 0x7f31d8108e00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f31c800b0a0 tx=0x7f31c800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 -- 172.21.15.37:0/1493610520 shutdown_connections 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 --2- 172.21.15.37:0/1493610520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d806b980 0x7f31d810f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 --2- 172.21.15.37:0/1493610520 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d8108a20 0x7f31d8108e00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 --2- 172.21.15.37:0/1493610520 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 -- 172.21.15.37:0/1493610520 >> 172.21.15.37:0/1493610520 conn(0x7f31d80788b0 msgr2=0x7f31d8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 -- 172.21.15.37:0/1493610520 shutdown_connections 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.007+0000 7f31dd400700 1 -- 172.21.15.37:0/1493610520 wait complete. 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.009+0000 7f31dd400700 1 Processor -- start 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 -- start start 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d806b980 0x7f31d819b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d8108a20 0x7f31d819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8112bb0 con 0x7f31d8108a20 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8112a30 con 0x7f31d806b980 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31dd400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f31d8112d30 con 0x7f31d806afe0 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31d6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31d6800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d819b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47902/0 (socket says 172.21.15.37:47902) 2024-04-26T01:56:21.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31d6800700 1 -- 172.21.15.37:0/3575257884 learned_addr learned my addr 172.21.15.37:0/3575257884 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.010+0000 7f31d7200700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d8108a20 0x7f31d819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31cde00700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d806b980 0x7f31d819b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d6800700 1 -- 172.21.15.37:0/3575257884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d806b980 msgr2=0x7f31d819b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d6800700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d806b980 0x7f31d819b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d6800700 1 -- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d8108a20 msgr2=0x7f31d819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d6800700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d8108a20 0x7f31d819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d6800700 1 -- 172.21.15.37:0/3575257884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31d81a0350 con 0x7f31d806afe0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d7200700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d8108a20 0x7f31d819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31cde00700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d806b980 0x7f31d819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.011+0000 7f31d6800700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d819b340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31c800b070 tx=0x7f31c8004d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.012+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c8021070 con 0x7f31d806afe0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.012+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31c8007e30 con 0x7f31d806afe0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.012+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31d81a05e0 con 0x7f31d806afe0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.012+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80046c0 con 0x7f31d806afe0 2024-04-26T01:56:21.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.012+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31d81a7ec0 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.013+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31d8069730 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.015+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f31c8007980 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.016+0000 7f31cfe00700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31bc0a3cc0 0x7f31bc0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.017+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f31c800f020 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.017+0000 7f31cde00700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31bc0a3cc0 0x7f31bc0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.017+0000 7f31cde00700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31bc0a3cc0 0x7f31bc0a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31d8078f00 tx=0x7f31c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.021+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31c802a020 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.259+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f31d802df80 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.259+0000 7f31cfe00700 1 -- 172.21.15.37:0/3575257884 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f31c809e1d0 con 0x7f31d806afe0 2024-04-26T01:56:21.110 INFO:teuthology.orchestra.run.smithi037.stdout:34359738474 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31bc0a3cc0 msgr2=0x7f31bc0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31bc0a3cc0 0x7f31bc0a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f31d8078f00 tx=0x7f31c0009260 comp rx=0 tx=0).stop 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 msgr2=0x7f31d819b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d819b340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f31c800b070 tx=0x7f31c8004d00 comp rx=0 tx=0).stop 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 shutdown_connections 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f31d8108a20 0x7f31d819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f31bc0a3cc0 0x7f31bc0a6180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f31d806afe0 0x7f31d819b340 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 --2- 172.21.15.37:0/3575257884 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f31d806b980 0x7f31d819b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 >> 172.21.15.37:0/3575257884 conn(0x7f31d80788b0 msgr2=0x7f31d810dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 shutdown_connections 2024-04-26T01:56:21.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:19.263+0000 7f31dd400700 1 -- 172.21.15.37:0/3575257884 wait complete. 2024-04-26T01:56:21.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.144+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f767002df80 con 0x7f7670104450 2024-04-26T01:56:21.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.145+0000 7f7667e00700 1 -- 172.21.15.37:0/1116012175 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f765809e250 con 0x7f7670104450 2024-04-26T01:56:21.144 INFO:teuthology.orchestra.run.smithi037.stdout:146028888148 2024-04-26T01:56:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f76540a3c40 msgr2=0x7f76540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f76540a3c40 0x7f76540a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7660009c60 tx=0x7f7660009260 comp rx=0 tx=0).stop 2024-04-26T01:56:21.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 msgr2=0x7f7670101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f7670101cd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f765800d010 tx=0x7f7658004d00 comp rx=0 tx=0).stop 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 shutdown_connections 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f76540a3c40 0x7f76540a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f767010cf60 0x7f7670102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7670103b30 0x7f7670101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 --2- 172.21.15.37:0/1116012175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7670104450 0x7f7670101cd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 >> 172.21.15.37:0/1116012175 conn(0x7f76700fd360 msgr2=0x7f76700fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 shutdown_connections 2024-04-26T01:56:21.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:21.148+0000 7f766fe00700 1 -- 172.21.15.37:0/1116012175 wait complete. 2024-04-26T01:56:21.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:20 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4097133684' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T01:56:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 -- 172.21.15.37:0/2617434775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 msgr2=0x7efc4410dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 --2- 172.21.15.37:0/2617434775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc4410dcd0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7efc300099c0 tx=0x7efc30009cd0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 -- 172.21.15.37:0/2617434775 shutdown_connections 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 --2- 172.21.15.37:0/2617434775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc441092d0 0x7efc441096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 --2- 172.21.15.37:0/2617434775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc4410e3b0 0x7efc44115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 --2- 172.21.15.37:0/2617434775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc4410dcd0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 -- 172.21.15.37:0/2617434775 >> 172.21.15.37:0/2617434775 conn(0x7efc44076210 msgr2=0x7efc44078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 -- 172.21.15.37:0/2617434775 shutdown_connections 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.934+0000 7efc4be00700 1 -- 172.21.15.37:0/2617434775 wait complete. 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.935+0000 7efc4be00700 1 Processor -- start 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 -- start start 2024-04-26T01:56:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc441092d0 0x7efc441a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc441a5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc4410e3b0 0x7efc4419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc44119270 con 0x7efc4410e3b0 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7efc441190f0 con 0x7efc441092d0 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.936+0000 7efc4be00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7efc441193f0 con 0x7efc44109c80 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc48a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc441a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc441092d0 0x7efc441a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc4410e3b0 0x7efc4419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc48a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc441a5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:47854/0 (socket says 172.21.15.37:47854) 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc48a00700 1 -- 172.21.15.37:0/379461999 learned_addr learned my addr 172.21.15.37:0/379461999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 -- 172.21.15.37:0/379461999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 msgr2=0x7efc441a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc441a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 -- 172.21.15.37:0/379461999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc441092d0 msgr2=0x7efc441a5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc441092d0 0x7efc441a5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 -- 172.21.15.37:0/379461999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4419fdb0 con 0x7efc4410e3b0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49400700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc441092d0 0x7efc441a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc48a00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc441a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.937+0000 7efc49e00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc4410e3b0 0x7efc4419f650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e970 tx=0x7efc3c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.938+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c004220 con 0x7efc4410e3b0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.938+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc3c0043c0 con 0x7efc4410e3b0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.938+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc441a00a0 con 0x7efc4410e3b0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.938+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c010700 con 0x7efc4410e3b0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.938+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc441ac310 con 0x7efc4410e3b0 2024-04-26T01:56:21.721 INFO:teuthology.orchestra.run.smithi037.stdout:167503724623 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.939+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc44069730 con 0x7efc4410e3b0 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.942+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7efc3c004750 con 0x7efc4410e3b0 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.944+0000 7efc42000700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efc280a3cc0 0x7efc280a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.944+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7efc3c0d0130 con 0x7efc4410e3b0 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.944+0000 7efc49400700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efc280a3cc0 0x7efc280a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.945+0000 7efc49400700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efc280a3cc0 0x7efc280a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc34009480 tx=0x7efc3400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:16.948+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc3c09cac0 con 0x7efc4410e3b0 2024-04-26T01:56:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.200+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7efc441a0b30 con 0x7efc4410e3b0 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.201+0000 7efc42000700 1 -- 172.21.15.37:0/379461999 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7efc3c09c3c0 con 0x7efc4410e3b0 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efc280a3cc0 msgr2=0x7efc280a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efc280a3cc0 0x7efc280a6180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efc34009480 tx=0x7efc3400b040 comp rx=0 tx=0).stop 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc4410e3b0 msgr2=0x7efc4419f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc4410e3b0 0x7efc4419f650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00e970 tx=0x7efc3c00ec80 comp rx=0 tx=0).stop 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 shutdown_connections 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc4410e3b0 0x7efc4419f650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7efc280a3cc0 0x7efc280a6180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7efc441092d0 0x7efc441a5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 --2- 172.21.15.37:0/379461999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7efc44109c80 0x7efc441a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 >> 172.21.15.37:0/379461999 conn(0x7efc44076210 msgr2=0x7efc44077600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 shutdown_connections 2024-04-26T01:56:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:17.204+0000 7efc4be00700 1 -- 172.21.15.37:0/379461999 wait complete. 2024-04-26T01:56:22.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:21 smithi045 ceph-mon[32520]: pgmap v400: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:22.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:21 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1116012175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:56:22.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:21 smithi037 ceph-mon[27906]: pgmap v400: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:22.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:21 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1116012175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:56:22.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:21 smithi071 ceph-mon[32578]: pgmap v400: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:22.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:21 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1116012175' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T01:56:22.531 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574948 for osd.1 2024-04-26T01:56:22.531 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:22.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.528+0000 7f557f800700 1 -- 172.21.15.37:0/1327182938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578074c70 msgr2=0x7f5578075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:22.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 --2- 172.21.15.37:0/1327182938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578074c70 0x7f5578075050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f556c009990 tx=0x7f556c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 -- 172.21.15.37:0/1327182938 shutdown_connections 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 --2- 172.21.15.37:0/1327182938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578074c70 0x7f5578075050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 --2- 172.21.15.37:0/1327182938 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f557810f470 0x7f5578111860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 --2- 172.21.15.37:0/1327182938 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578075590 0x7f557810ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 -- 172.21.15.37:0/1327182938 >> 172.21.15.37:0/1327182938 conn(0x7f5578079b10 msgr2=0x7f5578079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 -- 172.21.15.37:0/1327182938 shutdown_connections 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.529+0000 7f557f800700 1 -- 172.21.15.37:0/1327182938 wait complete. 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.530+0000 7f557f800700 1 Processor -- start 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.530+0000 7f557f800700 1 -- start start 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557f800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578074c70 0x7f557819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557f800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 0x7f557819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557f800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f557810f470 0x7f557819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:22.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557f800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5578114e70 con 0x7f557810f470 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557f800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5578114cf0 con 0x7f5578074c70 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557f800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5578114ff0 con 0x7f5578075590 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f5577e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 0x7f557819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f557d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f557810f470 0x7f557819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f5577e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 0x7f557819b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54984/0 (socket says 172.21.15.37:54984) 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f5577e00700 1 -- 172.21.15.37:0/4178163244 learned_addr learned my addr 172.21.15.37:0/4178163244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.531+0000 7f5577e00700 1 -- 172.21.15.37:0/4178163244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578074c70 msgr2=0x7f557819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5577e00700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578074c70 0x7f557819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5577e00700 1 -- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f557810f470 msgr2=0x7f557819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5577e00700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f557810f470 0x7f557819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5577e00700 1 -- 172.21.15.37:0/4178163244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55781a0370 con 0x7f5578075590 2024-04-26T01:56:22.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f557d800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f557810f470 0x7f557819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5577e00700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 0x7f557819b8a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f556400e970 tx=0x7f556400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556400cbd0 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55781a0660 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55640044d0 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564003de0 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.532+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55781a7ee0 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.533+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5578069730 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.535+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5564003f80 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.537+0000 7f5575600700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f55600a3c40 0x7f55600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.537+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f55640d0690 con 0x7f5578075590 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.537+0000 7f557ce00700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f55600a3c40 0x7f55600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.538+0000 7f557ce00700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f55600a3c40 0x7f55600a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f556c0098e0 tx=0x7f556c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:22.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.541+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f556409d020 con 0x7f5578075590 2024-04-26T01:56:22.683 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037814 got 257698037820 for osd.8 2024-04-26T01:56:22.683 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:22.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.801+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f55781a8140 con 0x7f5578075590 2024-04-26T01:56:22.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.801+0000 7f5575600700 1 -- 172.21.15.37:0/4178163244 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f556409c920 con 0x7f5578075590 2024-04-26T01:56:22.801 INFO:teuthology.orchestra.run.smithi037.stdout:313532612659 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.804+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f55600a3c40 msgr2=0x7f55600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.804+0000 7f557f800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f55600a3c40 0x7f55600a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f556c0098e0 tx=0x7f556c005fb0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 msgr2=0x7f557819b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 0x7f557819b8a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f556400e970 tx=0x7f556400ed30 comp rx=0 tx=0).stop 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 shutdown_connections 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f557810f470 0x7f557819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f55600a3c40 0x7f55600a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5578074c70 0x7f557819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 --2- 172.21.15.37:0/4178163244 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5578075590 0x7f557819b8a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 >> 172.21.15.37:0/4178163244 conn(0x7f5578079b10 msgr2=0x7f5578100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 shutdown_connections 2024-04-26T01:56:22.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:22.805+0000 7f557f800700 1 -- 172.21.15.37:0/4178163244 wait complete. 2024-04-26T01:56:23.589 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149778 got 111669149784 for osd.3 2024-04-26T01:56:23.589 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:23.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:23 smithi037 ceph-mon[27906]: pgmap v401: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:23.950 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:23 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4178163244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:56:24.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:23 smithi045 ceph-mon[32520]: pgmap v401: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:24.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:23 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4178163244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:56:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:24.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:23 smithi071 ceph-mon[32578]: pgmap v401: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:24.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:23 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4178163244' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T01:56:24.672 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738467 got 34359738474 for osd.0 2024-04-26T01:56:24.673 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:25 smithi045 ceph-mon[32520]: pgmap v402: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:26.106 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:25 smithi037 ceph-mon[27906]: pgmap v402: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:25 smithi071 ceph-mon[32578]: pgmap v402: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:26.218 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888143 got 146028888148 for osd.4 2024-04-26T01:56:26.218 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:26.520 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411424 for osd.2 2024-04-26T01:56:26.520 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:26.673 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351016 got 347892351022 for osd.11 2024-04-26T01:56:26.673 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:27.244 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612653 got 313532612659 for osd.10 2024-04-26T01:56:27.244 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:27.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.465+0000 7f2b8d000700 1 -- 172.21.15.37:0/640011628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b88108a30 msgr2=0x7f2b88108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.465+0000 7f2b8d000700 1 --2- 172.21.15.37:0/640011628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b88108a30 0x7f2b88108e10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009f90 tx=0x7f2b7800b330 comp rx=0 tx=0).stop 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 -- 172.21.15.37:0/640011628 shutdown_connections 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 --2- 172.21.15.37:0/640011628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b88108a30 0x7f2b88108e10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 --2- 172.21.15.37:0/640011628 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b8806b0f0 0x7f2b8810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 --2- 172.21.15.37:0/640011628 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b8806a750 0x7f2b8806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 -- 172.21.15.37:0/640011628 >> 172.21.15.37:0/640011628 conn(0x7f2b880788f0 msgr2=0x7f2b88078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 -- 172.21.15.37:0/640011628 shutdown_connections 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.466+0000 7f2b8d000700 1 -- 172.21.15.37:0/640011628 wait complete. 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 Processor -- start 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 -- start start 2024-04-26T01:56:27.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 0x7f2b88101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b8806b0f0 0x7f2b88101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b88108a30 0x7f2b88102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88112bb0 con 0x7f2b8806a750 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88112a30 con 0x7f2b88108a30 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.467+0000 7f2b8d000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2b88112d30 con 0x7f2b8806b0f0 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 0x7f2b88101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 0x7f2b88101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35780/0 (socket says 172.21.15.37:35780) 2024-04-26T01:56:27.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 -- 172.21.15.37:0/1825094431 learned_addr learned my addr 172.21.15.37:0/1825094431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stdout:201863462986 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b7fe00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b8806b0f0 0x7f2b88101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b86800700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b88108a30 0x7f2b88102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 -- 172.21.15.37:0/1825094431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b8806b0f0 msgr2=0x7f2b88101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b8806b0f0 0x7f2b88101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 -- 172.21.15.37:0/1825094431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b88108a30 msgr2=0x7f2b88102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:27.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b88108a30 0x7f2b88102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 -- 172.21.15.37:0/1825094431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b881a7610 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.468+0000 7f2b85e00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 0x7f2b88101a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009f60 tx=0x7f2b7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.469+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78021070 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.469+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b78007ac0 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.469+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b78005040 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.469+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b881a78a0 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.469+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b881a7d00 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.470+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b88069730 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.472+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2b78024070 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.474+0000 7f2b7ea00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b6c0a3cc0 0x7f2b6c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.475+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f2b7800f020 con 0x7f2b8806a750 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.475+0000 7f2b7fe00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b6c0a3cc0 0x7f2b6c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:27.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.475+0000 7f2b7fe00700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b6c0a3cc0 0x7f2b6c0a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009750 tx=0x7f2b70009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.480+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b7802b020 con 0x7f2b8806a750 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.718+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2b8802e140 con 0x7f2b8806a750 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.719+0000 7f2b7ea00700 1 -- 172.21.15.37:0/1825094431 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2b7809e180 con 0x7f2b8806a750 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b6c0a3cc0 msgr2=0x7f2b6c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b6c0a3cc0 0x7f2b6c0a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009750 tx=0x7f2b70009310 comp rx=0 tx=0).stop 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 msgr2=0x7f2b88101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 0x7f2b88101a10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2b78009f60 tx=0x7f2b7800ccb0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 shutdown_connections 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2b8806a750 0x7f2b88101a10 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b6c0a3cc0 0x7f2b6c0a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2b88108a30 0x7f2b88102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 --2- 172.21.15.37:0/1825094431 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2b8806b0f0 0x7f2b88101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.722+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 >> 172.21.15.37:0/1825094431 conn(0x7f2b880788f0 msgr2=0x7f2b880ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:27.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.723+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 shutdown_connections 2024-04-26T01:56:27.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:27.723+0000 7f2b8d000700 1 -- 172.21.15.37:0/1825094431 wait complete. 2024-04-26T01:56:27.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724617 got 167503724623 for osd.5 2024-04-26T01:56:27.970 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:27 smithi045 ceph-mon[32520]: pgmap v403: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T01:56:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:27 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1825094431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:56:28.004 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:27 smithi037 ceph-mon[27906]: pgmap v403: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T01:56:28.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:27 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1825094431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:56:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:27 smithi071 ceph-mon[32578]: pgmap v403: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T01:56:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:27 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1825094431' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T01:56:29.153 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462978 got 201863462986 for osd.6 2024-04-26T01:56:29.153 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:29.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 -- 172.21.15.37:0/1112352624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c104db0 msgr2=0x7f039c105190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 --2- 172.21.15.37:0/1112352624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c104db0 0x7f039c105190 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0390009990 tx=0x7f0390009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 -- 172.21.15.37:0/1112352624 shutdown_connections 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 --2- 172.21.15.37:0/1112352624 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039c109e90 0x7f039c111a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 --2- 172.21.15.37:0/1112352624 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c104db0 0x7f039c105190 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 --2- 172.21.15.37:0/1112352624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c105760 0x7f039c1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 -- 172.21.15.37:0/1112352624 >> 172.21.15.37:0/1112352624 conn(0x7f039c100680 msgr2=0x7f039c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 -- 172.21.15.37:0/1112352624 shutdown_connections 2024-04-26T01:56:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.790+0000 7f03a2400700 1 -- 172.21.15.37:0/1112352624 wait complete. 2024-04-26T01:56:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.791+0000 7f03a2400700 1 Processor -- start 2024-04-26T01:56:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 -- start start 2024-04-26T01:56:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c104db0 0x7f039c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 0x7f039c19b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039c109e90 0x7f039c19fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:29.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f039c114d40 con 0x7f039c104db0 2024-04-26T01:56:29.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f039c114bc0 con 0x7f039c105760 2024-04-26T01:56:29.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.792+0000 7f03a2400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f039c114ec0 con 0x7f039c109e90 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 0x7f039c19b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 0x7f039c19b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:46788/0 (socket says 172.21.15.37:46788) 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f03a1a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039c109e90 0x7f039c19fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 -- 172.21.15.37:0/683713099 learned_addr learned my addr 172.21.15.37:0/683713099 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f03a1000700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c104db0 0x7f039c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 -- 172.21.15.37:0/683713099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039c109e90 msgr2=0x7f039c19fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039c109e90 0x7f039c19fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 -- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c104db0 msgr2=0x7f039c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:29.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c104db0 0x7f039c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 -- 172.21.15.37:0/683713099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f039c1a0280 con 0x7f039c105760 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f03a1000700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c104db0 0x7f039c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.793+0000 7f039be00700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 0x7f039c19b7b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f038800e970 tx=0x7f038800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.794+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f038800cbd0 con 0x7f039c105760 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.794+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f038800cd70 con 0x7f039c105760 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.794+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0388005680 con 0x7f039c105760 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.794+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f039c1a0570 con 0x7f039c105760 2024-04-26T01:56:29.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.794+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f039c1a7df0 con 0x7f039c105760 2024-04-26T01:56:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.795+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f039c069730 con 0x7f039c105760 2024-04-26T01:56:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.797+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0388004750 con 0x7f039c105760 2024-04-26T01:56:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.798+0000 7f0399600700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f03840a3cc0 0x7f03840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:29.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.798+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f03880d0790 con 0x7f039c105760 2024-04-26T01:56:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.799+0000 7f03a1000700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f03840a3cc0 0x7f03840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:29.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.799+0000 7f03a1000700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f03840a3cc0 0x7f03840a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f039000b2e0 tx=0x7f039001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:29.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:29.803+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f038809d120 con 0x7f039c105760 2024-04-26T01:56:29.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:29 smithi037 ceph-mon[27906]: pgmap v404: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:30.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:29 smithi045 ceph-mon[32520]: pgmap v404: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:30.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.054+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f039c02df80 con 0x7f039c105760 2024-04-26T01:56:30.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.055+0000 7f0399600700 1 -- 172.21.15.37:0/683713099 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f038809ca20 con 0x7f039c105760 2024-04-26T01:56:30.054 INFO:teuthology.orchestra.run.smithi037.stdout:292057776185 2024-04-26T01:56:30.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f03840a3cc0 msgr2=0x7f03840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f03840a3cc0 0x7f03840a6180 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f039000b2e0 tx=0x7f039001a040 comp rx=0 tx=0).stop 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 msgr2=0x7f039c19b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 0x7f039c19b7b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f038800e970 tx=0x7f038800ec80 comp rx=0 tx=0).stop 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 shutdown_connections 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f039c104db0 0x7f039c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f03840a3cc0 0x7f03840a6180 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f039c109e90 0x7f039c19fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 --2- 172.21.15.37:0/683713099 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f039c105760 0x7f039c19b7b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 >> 172.21.15.37:0/683713099 conn(0x7f039c100680 msgr2=0x7f039c109020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.058+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 shutdown_connections 2024-04-26T01:56:30.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:30.059+0000 7f03a2400700 1 -- 172.21.15.37:0/683713099 wait complete. 2024-04-26T01:56:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:29 smithi071 ceph-mon[32578]: pgmap v404: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:56:31.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:30 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/683713099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:56:31.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:30 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/683713099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:56:31.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:30 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/683713099' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T01:56:31.316 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776179 got 292057776185 for osd.9 2024-04-26T01:56:31.317 DEBUG:teuthology.parallel:result is None 2024-04-26T01:56:31.317 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:56:31.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:56:31.803 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:32.111 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:31 smithi037 ceph-mon[27906]: pgmap v405: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:32.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:31 smithi071 ceph-mon[32578]: pgmap v405: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:31 smithi045 ceph-mon[32520]: pgmap v405: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:33.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.979+0000 7f38a8200700 1 -- 172.21.15.37:0/1664519984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a0108a30 msgr2=0x7f38a0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.979+0000 7f38a8200700 1 --2- 172.21.15.37:0/1664519984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a0108a30 0x7f38a0108e10 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f38940098f0 tx=0x7f3894009c00 comp rx=0 tx=0).stop 2024-04-26T01:56:33.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 -- 172.21.15.37:0/1664519984 shutdown_connections 2024-04-26T01:56:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 --2- 172.21.15.37:0/1664519984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a0108a30 0x7f38a0108e10 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 --2- 172.21.15.37:0/1664519984 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a006b0f0 0x7f38a010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 --2- 172.21.15.37:0/1664519984 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 -- 172.21.15.37:0/1664519984 >> 172.21.15.37:0/1664519984 conn(0x7f38a00788f0 msgr2=0x7f38a0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 -- 172.21.15.37:0/1664519984 shutdown_connections 2024-04-26T01:56:33.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.980+0000 7f38a8200700 1 -- 172.21.15.37:0/1664519984 wait complete. 2024-04-26T01:56:33.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.981+0000 7f38a8200700 1 Processor -- start 2024-04-26T01:56:33.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.981+0000 7f38a8200700 1 -- start start 2024-04-26T01:56:33.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a8200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a8200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 0x7f38a01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a8200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0108a30 0x7f38a019da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a8200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0112c20 con 0x7f38a006b0f0 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a8200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0112aa0 con 0x7f38a006a750 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a8200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f38a0112da0 con 0x7f38a0108a30 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 0x7f38a01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a6200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0108a30 0x7f38a019da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a4e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 0x7f38a01996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40580/0 (socket says 172.21.15.37:40580) 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a5800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a0199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:41462/0 (socket says 172.21.15.37:41462) 2024-04-26T01:56:33.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a4e00700 1 -- 172.21.15.37:0/2645521711 learned_addr learned my addr 172.21.15.37:0/2645521711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.982+0000 7f38a6200700 1 -- 172.21.15.37:0/2645521711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 msgr2=0x7f38a0199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a6200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a0199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a6200700 1 -- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 msgr2=0x7f38a01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a6200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 0x7f38a01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a6200700 1 -- 172.21.15.37:0/2645521711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38a019e1a0 con 0x7f38a0108a30 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a5800700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a0199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:33.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a4e00700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 0x7f38a01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a6200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0108a30 0x7f38a019da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f389800efc0 tx=0x7f389800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38980092e0 con 0x7f38a0108a30 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3898004bb0 con 0x7f38a0108a30 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3898004020 con 0x7f38a0108a30 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38a019e430 con 0x7f38a0108a30 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.983+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38a01a5d10 con 0x7f38a0108a30 2024-04-26T01:56:33.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.985+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38a0069730 con 0x7f38a0108a30 2024-04-26T01:56:33.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.987+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f389801b030 con 0x7f38a0108a30 2024-04-26T01:56:33.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.988+0000 7f389e000700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f38880a3cc0 0x7f38880a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:33.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.988+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f38980cfdc0 con 0x7f38a0108a30 2024-04-26T01:56:33.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.988+0000 7f38a5800700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f38880a3cc0 0x7f38880a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:33.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.989+0000 7f38a5800700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f38880a3cc0 0x7f38880a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38940098c0 tx=0x7f3894019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:33.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:33.993+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3898022020 con 0x7f38a0108a30 2024-04-26T01:56:34.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:33 smithi071 ceph-mon[32578]: pgmap v406: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:34.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:33 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:34.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:33 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:34.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:33 smithi037 ceph-mon[27906]: pgmap v406: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:34.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:33 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:34.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:33 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:34.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.230+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38a0062e80 con 0x7f38880a3cc0 2024-04-26T01:56:34.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.240+0000 7f389e000700 1 -- 172.21.15.37:0/2645521711 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7f38a0062e80 con 0x7f38880a3cc0 2024-04-26T01:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f38880a3cc0 msgr2=0x7f38880a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f38880a3cc0 0x7f38880a6180 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f38940098c0 tx=0x7f3894019040 comp rx=0 tx=0).stop 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0108a30 msgr2=0x7f38a019da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0108a30 0x7f38a019da60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f389800efc0 tx=0x7f389800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 shutdown_connections 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f38880a3cc0 0x7f38880a6180 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f38a0108a30 0x7f38a019da60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f38a006a750 0x7f38a0199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 --2- 172.21.15.37:0/2645521711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f38a006b0f0 0x7f38a01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 >> 172.21.15.37:0/2645521711 conn(0x7f38a00788f0 msgr2=0x7f38a00ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 shutdown_connections 2024-04-26T01:56:34.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:34.248+0000 7f38a8200700 1 -- 172.21.15.37:0/2645521711 wait complete. 2024-04-26T01:56:34.249 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:56:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:33 smithi045 ceph-mon[32520]: pgmap v406: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:33 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:33 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:34.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:34 smithi037 ceph-mon[27906]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:56:34.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:34.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:34.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:34 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-04-26T01:56:32.575684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653984,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":398,"num_read_kb":314,"num_write":271,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371032,"kb_used_data":13560,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570792,"statfs":{"total":1151940427776,"available":1151560491008,"internally_reserved":0,"allocated":13885440,"data_stored":7757196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005645"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524517+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:56:12.524517+0000","last_peered":"2024-04-26T01:56:12.524517+0000","last_clean":"2024-04-26T01:56:12.524517+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:56:12.524517+0000","last_undegraded":"2024-04-26T01:56:12.524517+0000","last_fullsized":"2024-04-26T01:56:12.524517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519684+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:56:12.519684+0000","last_peered":"2024-04-26T01:56:12.519684+0000","last_clean":"2024-04-26T01:56:12.519684+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:56:12.519684+0000","last_undegraded":"2024-04-26T01:56:12.519684+0000","last_fullsized":"2024-04-26T01:56:12.519684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528221+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:56:12.528221+0000","last_peered":"2024-04-26T01:56:12.528221+0000","last_clean":"2024-04-26T01:56:12.528221+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:56:12.528221+0000","last_undegraded":"2024-04-26T01:56:12.528221+0000","last_fullsized":"2024-04-26T01:56:12.528221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.504394+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:56:12.504394+0000","last_peered":"2024-04-26T01:56:12.504394+0000","last_clean":"2024-04-26T01:56:12.504394+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:56:12.504394+0000","last_undegraded":"2024-04-26T01:56:12.504394+0000","last_fullsized":"2024-04-26T01:56:12.504394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525240+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:56:12.525240+0000","last_peered":"2024-04-26T01:56:12.525240+0000","last_clean":"2024-04-26T01:56:12.525240+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:56:12.525240+0000","last_undegraded":"2024-04-26T01:56:12.525240+0000","last_fullsized":"2024-04-26T01:56:12.525240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.506176+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:56:12.506176+0000","last_peered":"2024-04-26T01:56:12.506176+0000","last_clean":"2024-04-26T01:56:12.506176+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:56:12.506176+0000","last_undegraded":"2024-04-26T01:56:12.506176+0000","last_fullsized":"2024-04-26T01:56:12.506176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.502939+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:56:12.502939+0000","last_peered":"2024-04-26T01:56:12.502939+0000","last_clean":"2024-04-26T01:56:12.502939+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:56:12.502939+0000","last_undegraded":"2024-04-26T01:56:12.502939+0000","last_fullsized":"2024-04-26T01:56:12.502939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528586+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:56:12.528586+0000","last_peered":"2024-04-26T01:56:12.528586+0000","last_clean":"2024-04-26T01:56:12.528586+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:56:12.528586+0000","last_undegraded":"2024-04-26T01:56:12.528586+0000","last_fullsized":"2024-04-26T01:56:12.528586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.523595+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:56:12.523595+0000","last_peered":"2024-04-26T01:56:12.523595+0000","last_clean":"2024-04-26T01:56:12.523595+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:56:12.523595+0000","last_undegraded":"2024-04-26T01:56:12.523595+0000","last_fullsized":"2024-04-26T01:56:12.523595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.511960+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:56:12.511960+0000","last_peered":"2024-04-26T01:56:12.511960+0000","last_clean":"2024-04-26T01:56:12.511960+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:56:12.511960+0000","last_undegraded":"2024-04-26T01:56:12.511960+0000","last_fullsized":"2024-04-26T01:56:12.511960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.520329+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:56:12.520329+0000","last_peered":"2024-04-26T01:56:12.520329+0000","last_clean":"2024-04-26T01:56:12.520329+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:56:12.520329+0000","last_undegraded":"2024-04-26T01:56:12.520329+0000","last_fullsized":"2024-04-26T01:56:12.520329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.509394+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:56:12.509394+0000","last_peered":"2024-04-26T01:56:12.509394+0000","last_clean":"2024-04-26T01:56:12.509394+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:56:12.509394+0000","last_undegraded":"2024-04-26T01:56:12.509394+0000","last_fullsized":"2024-04-26T01:56:12.509394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.511929+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:56:12.511929+0000","last_peered":"2024-04-26T01:56:12.511929+0000","last_clean":"2024-04-26T01:56:12.511929+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:56:12.511929+0000","last_undegraded":"2024-04-26T01:56:12.511929+0000","last_fullsized":"2024-04-26T01:56:12.511929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521178+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:56:12.521178+0000","last_peered":"2024-04-26T01:56:12.521178+0000","last_clean":"2024-04-26T01:56:12.521178+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:56:12.521178+0000","last_undegraded":"2024-04-26T01:56:12.521178+0000","last_fullsized":"2024-04-26T01:56:12.521178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.522934+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:56:12.522934+0000","last_peered":"2024-04-26T01:56:12.522934+0000","last_clean":"2024-04-26T01:56:12.522934+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:56:12.522934+0000","last_undegraded":"2024-04-26T01:56:12.522934+0000","last_fullsized":"2024-04-26T01:56:12.522934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.523067+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:56:12.523067+0000","last_peered":"2024-04-26T01:56:12.523067+0000","last_clean":"2024-04-26T01:56:12.523067+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:56:12.523067+0000","last_undegraded":"2024-04-26T01:56:12.523067+0000","last_fullsized":"2024-04-26T01:56:12.523067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524424+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:56:12.524424+0000","last_peered":"2024-04-26T01:56:12.524424+0000","last_clean":"2024-04-26T01:56:12.524424+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:56:12.524424+0000","last_undegraded":"2024-04-26T01:56:12.524424+0000","last_fullsized":"2024-04-26T01:56:12.524424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519455+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:56:12.519455+0000","last_peered":"2024-04-26T01:56:12.519455+0000","last_clean":"2024-04-26T01:56:12.519455+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:56:12.519455+0000","last_undegraded":"2024-04-26T01:56:12.519455+0000","last_fullsized":"2024-04-26T01:56:12.519455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529406+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:56:12.529406+0000","last_peered":"2024-04-26T01:56:12.529406+0000","last_clean":"2024-04-26T01:56:12.529406+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:56:12.529406+0000","last_undegraded":"2024-04-26T01:56:12.529406+0000","last_fullsized":"2024-04-26T01:56:12.529406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.187845+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:54:44.187845+0000","last_peered":"2024-04-26T01:54:44.187845+0000","last_clean":"2024-04-26T01:54:44.187845+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:54:44.187845+0000","last_undegraded":"2024-04-26T01:54:44.187845+0000","last_fullsized":"2024-04-26T01:54:44.187845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529646+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:56:12.529646+0000","last_peered":"2024-04-26T01:56:12.529646+0000","last_clean":"2024-04-26T01:56:12.529646+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:56:12.529646+0000","last_undegraded":"2024-04-26T01:56:12.529646+0000","last_fullsized":"2024-04-26T01:56:12.529646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.184671+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:54:44.184671+0000","last_peered":"2024-04-26T01:54:44.184671+0000","last_clean":"2024-04-26T01:54:44.184671+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:54:44.184671+0000","last_undegraded":"2024-04-26T01:54:44.184671+0000","last_fullsized":"2024-04-26T01:54:44.184671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528040+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:56:12.528040+0000","last_peered":"2024-04-26T01:56:12.528040+0000","last_clean":"2024-04-26T01:56:12.528040+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:56:12.528040+0000","last_undegraded":"2024-04-26T01:56:12.528040+0000","last_fullsized":"2024-04-26T01:56:12.528040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178449+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:54:44.178449+0000","last_peered":"2024-04-26T01:54:44.178449+0000","last_clean":"2024-04-26T01:54:44.178449+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:54:44.178449+0000","last_undegraded":"2024-04-26T01:54:44.178449+0000","last_fullsized":"2024-04-26T01:54:44.178449+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521140+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:56:12.521140+0000","last_peered":"2024-04-26T01:56:12.521140+0000","last_clean":"2024-04-26T01:56:12.521140+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:56:12.521140+0000","last_undegraded":"2024-04-26T01:56:12.521140+0000","last_fullsized":"2024-04-26T01:56:12.521140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178726+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:54:44.178726+0000","last_peered":"2024-04-26T01:54:44.178726+0000","last_clean":"2024-04-26T01:54:44.178726+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:54:44.178726+0000","last_undegraded":"2024-04-26T01:54:44.178726+0000","last_fullsized":"2024-04-26T01:54:44.178726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.510658+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:56:12.510658+0000","last_peered":"2024-04-26T01:56:12.510658+0000","last_clean":"2024-04-26T01:56:12.510658+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:56:12.510658+0000","last_undegraded":"2024-04-26T01:56:12.510658+0000","last_fullsized":"2024-04-26T01:56:12.510658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374980+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:54:44.374980+0000","last_peered":"2024-04-26T01:54:44.374980+0000","last_clean":"2024-04-26T01:54:44.374980+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:54:44.374980+0000","last_undegraded":"2024-04-26T01:54:44.374980+0000","last_fullsized":"2024-04-26T01:54:44.374980+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.502283+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:56:12.502283+0000","last_peered":"2024-04-26T01:56:12.502283+0000","last_clean":"2024-04-26T01:56:12.502283+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:56:12.502283+0000","last_undegraded":"2024-04-26T01:56:12.502283+0000","last_fullsized":"2024-04-26T01:56:12.502283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375951+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:54:44.375951+0000","last_peered":"2024-04-26T01:54:44.375951+0000","last_clean":"2024-04-26T01:54:44.375951+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:54:44.375951+0000","last_undegraded":"2024-04-26T01:54:44.375951+0000","last_fullsized":"2024-04-26T01:54:44.375951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525262+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:56:12.525262+0000","last_peered":"2024-04-26T01:56:12.525262+0000","last_clean":"2024-04-26T01:56:12.525262+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:56:12.525262+0000","last_undegraded":"2024-04-26T01:56:12.525262+0000","last_fullsized":"2024-04-26T01:56:12.525262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376752+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:54:44.376752+0000","last_peered":"2024-04-26T01:54:44.376752+0000","last_clean":"2024-04-26T01:54:44.376752+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:54:44.376752+0000","last_undegraded":"2024-04-26T01:54:44.376752+0000","last_fullsized":"2024-04-26T01:54:44.376752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529277+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:56:12.529277+0000","last_peered":"2024-04-26T01:56:12.529277+0000","last_clean":"2024-04-26T01:56:12.529277+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:56:12.529277+0000","last_undegraded":"2024-04-26T01:56:12.529277+0000","last_fullsized":"2024-04-26T01:56:12.529277+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375038+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:54:44.375038+0000","last_peered":"2024-04-26T01:54:44.375038+0000","last_clean":"2024-04-26T01:54:44.375038+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:54:44.375038+0000","last_undegraded":"2024-04-26T01:54:44.375038+0000","last_fullsized":"2024-04-26T01:54:44.375038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528978+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:56:12.528978+0000","last_peered":"2024-04-26T01:56:12.528978+0000","last_clean":"2024-04-26T01:56:12.528978+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:56:12.528978+0000","last_undegraded":"2024-04-26T01:56:12.528978+0000","last_fullsized":"2024-04-26T01:56:12.528978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376847+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:54:44.376847+0000","last_peered":"2024-04-26T01:54:44.376847+0000","last_clean":"2024-04-26T01:54:44.376847+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:54:44.376847+0000","last_undegraded":"2024-04-26T01:54:44.376847+0000","last_fullsized":"2024-04-26T01:54:44.376847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.520527+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:56:12.520527+0000","last_peered":"2024-04-26T01:56:12.520527+0000","last_clean":"2024-04-26T01:56:12.520527+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:56:12.520527+0000","last_undegraded":"2024-04-26T01:56:12.520527+0000","last_fullsized":"2024-04-26T01:56:12.520527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377265+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:54:44.377265+0000","last_peered":"2024-04-26T01:54:44.377265+0000","last_clean":"2024-04-26T01:54:44.377265+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:54:44.377265+0000","last_undegraded":"2024-04-26T01:54:44.377265+0000","last_fullsized":"2024-04-26T01:54:44.377265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.516160+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:56:12.516160+0000","last_peered":"2024-04-26T01:56:12.516160+0000","last_clean":"2024-04-26T01:56:12.516160+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:56:12.516160+0000","last_undegraded":"2024-04-26T01:56:12.516160+0000","last_fullsized":"2024-04-26T01:56:12.516160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375230+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:54:44.375230+0000","last_peered":"2024-04-26T01:54:44.375230+0000","last_clean":"2024-04-26T01:54:44.375230+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:54:44.375230+0000","last_undegraded":"2024-04-26T01:54:44.375230+0000","last_fullsized":"2024-04-26T01:54:44.375230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524424+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:56:12.524424+0000","last_peered":"2024-04-26T01:56:12.524424+0000","last_clean":"2024-04-26T01:56:12.524424+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:56:12.524424+0000","last_undegraded":"2024-04-26T01:56:12.524424+0000","last_fullsized":"2024-04-26T01:56:12.524424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":314,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:31.720549+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:56:31.720549+0000","last_peered":"2024-04-26T01:56:31.720549+0000","last_clean":"2024-04-26T01:56:31.720549+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:56:31.720549+0000","last_undegraded":"2024-04-26T01:56:31.720549+0000","last_fullsized":"2024-04-26T01:56:31.720549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62584,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":256,"num_read_kb":189,"num_write":31,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.512660+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:56:12.512660+0000","last_peered":"2024-04-26T01:56:12.512660+0000","last_clean":"2024-04-26T01:56:12.512660+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:56:12.512660+0000","last_undegraded":"2024-04-26T01:56:12.512660+0000","last_fullsized":"2024-04-26T01:56:12.512660+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175227+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:54:44.175227+0000","last_peered":"2024-04-26T01:54:44.175227+0000","last_clean":"2024-04-26T01:54:44.175227+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:54:44.175227+0000","last_undegraded":"2024-04-26T01:54:44.175227+0000","last_fullsized":"2024-04-26T01:54:44.175227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529156+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:56:12.529156+0000","last_peered":"2024-04-26T01:56:12.529156+0000","last_clean":"2024-04-26T01:56:12.529156+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:56:12.529156+0000","last_undegraded":"2024-04-26T01:56:12.529156+0000","last_fullsized":"2024-04-26T01:56:12.529156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374977+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:54:44.374977+0000","last_peered":"2024-04-26T01:54:44.374977+0000","last_clean":"2024-04-26T01:54:44.374977+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:54:44.374977+0000","last_undegraded":"2024-04-26T01:54:44.374977+0000","last_fullsized":"2024-04-26T01:54:44.374977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519960+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:56:12.519960+0000","last_peered":"2024-04-26T01:56:12.519960+0000","last_clean":"2024-04-26T01:56:12.519960+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:56:12.519960+0000","last_undegraded":"2024-04-26T01:56:12.519960+0000","last_fullsized":"2024-04-26T01:56:12.519960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174994+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:54:44.174994+0000","last_peered":"2024-04-26T01:54:44.174994+0000","last_clean":"2024-04-26T01:54:44.174994+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:54:44.174994+0000","last_undegraded":"2024-04-26T01:54:44.174994+0000","last_fullsized":"2024-04-26T01:54:44.174994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.512438+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:56:12.512438+0000","last_peered":"2024-04-26T01:56:12.512438+0000","last_clean":"2024-04-26T01:56:12.512438+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:56:12.512438+0000","last_undegraded":"2024-04-26T01:56:12.512438+0000","last_fullsized":"2024-04-26T01:56:12.512438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.198548+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:54:44.198548+0000","last_peered":"2024-04-26T01:54:44.198548+0000","last_clean":"2024-04-26T01:54:44.198548+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:54:44.198548+0000","last_undegraded":"2024-04-26T01:54:44.198548+0000","last_fullsized":"2024-04-26T01:54:44.198548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.522989+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:56:12.522989+0000","last_peered":"2024-04-26T01:56:12.522989+0000","last_clean":"2024-04-26T01:56:12.522989+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:56:12.522989+0000","last_undegraded":"2024-04-26T01:56:12.522989+0000","last_fullsized":"2024-04-26T01:56:12.522989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178306+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:54:44.178306+0000","last_peered":"2024-04-26T01:54:44.178306+0000","last_clean":"2024-04-26T01:54:44.178306+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:54:44.178306+0000","last_undegraded":"2024-04-26T01:54:44.178306+0000","last_fullsized":"2024-04-26T01:54:44.178306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505709+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:56:12.505709+0000","last_peered":"2024-04-26T01:56:12.505709+0000","last_clean":"2024-04-26T01:56:12.505709+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:56:12.505709+0000","last_undegraded":"2024-04-26T01:56:12.505709+0000","last_fullsized":"2024-04-26T01:56:12.505709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.182950+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:54:44.182950+0000","last_peered":"2024-04-26T01:54:44.182950+0000","last_clean":"2024-04-26T01:54:44.182950+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:54:44.182950+0000","last_undegraded":"2024-04-26T01:54:44.182950+0000","last_fullsized":"2024-04-26T01:54:44.182950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.500966+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:56:12.500966+0000","last_peered":"2024-04-26T01:56:12.500966+0000","last_clean":"2024-04-26T01:56:12.500966+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:56:12.500966+0000","last_undegraded":"2024-04-26T01:56:12.500966+0000","last_fullsized":"2024-04-26T01:56:12.500966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376577+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:54:44.376577+0000","last_peered":"2024-04-26T01:54:44.376577+0000","last_clean":"2024-04-26T01:54:44.376577+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:54:44.376577+0000","last_undegraded":"2024-04-26T01:54:44.376577+0000","last_fullsized":"2024-04-26T01:54:44.376577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.511810+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:56:12.511810+0000","last_peered":"2024-04-26T01:56:12.511810+0000","last_clean":"2024-04-26T01:56:12.511810+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:56:12.511810+0000","last_undegraded":"2024-04-26T01:56:12.511810+0000","last_fullsized":"2024-04-26T01:56:12.511810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.496771+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:56:12.496771+0000","last_peered":"2024-04-26T01:56:12.496771+0000","last_clean":"2024-04-26T01:56:12.496771+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:56:12.496771+0000","last_undegraded":"2024-04-26T01:56:12.496771+0000","last_fullsized":"2024-04-26T01:56:12.496771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376367+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:54:44.376367+0000","last_peered":"2024-04-26T01:54:44.376367+0000","last_clean":"2024-04-26T01:54:44.376367+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:54:44.376367+0000","last_undegraded":"2024-04-26T01:54:44.376367+0000","last_fullsized":"2024-04-26T01:54:44.376367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528127+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:56:12.528127+0000","last_peered":"2024-04-26T01:56:12.528127+0000","last_clean":"2024-04-26T01:56:12.528127+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:56:12.528127+0000","last_undegraded":"2024-04-26T01:56:12.528127+0000","last_fullsized":"2024-04-26T01:56:12.528127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175097+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:54:44.175097+0000","last_peered":"2024-04-26T01:54:44.175097+0000","last_clean":"2024-04-26T01:54:44.175097+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:54:44.175097+0000","last_undegraded":"2024-04-26T01:54:44.175097+0000","last_fullsized":"2024-04-26T01:54:44.175097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376592+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:54:44.376592+0000","last_peered":"2024-04-26T01:54:44.376592+0000","last_clean":"2024-04-26T01:54:44.376592+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:54:44.376592+0000","last_undegraded":"2024-04-26T01:54:44.376592+0000","last_fullsized":"2024-04-26T01:54:44.376592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505430+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:56:12.505430+0000","last_peered":"2024-04-26T01:56:12.505430+0000","last_clean":"2024-04-26T01:56:12.505430+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:56:12.505430+0000","last_undegraded":"2024-04-26T01:56:12.505430+0000","last_fullsized":"2024-04-26T01:56:12.505430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376779+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:54:44.376779+0000","last_peered":"2024-04-26T01:54:44.376779+0000","last_clean":"2024-04-26T01:54:44.376779+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:54:44.376779+0000","last_undegraded":"2024-04-26T01:54:44.376779+0000","last_fullsized":"2024-04-26T01:54:44.376779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178343+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:54:44.178343+0000","last_peered":"2024-04-26T01:54:44.178343+0000","last_clean":"2024-04-26T01:54:44.178343+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:54:44.178343+0000","last_undegraded":"2024-04-26T01:54:44.178343+0000","last_fullsized":"2024-04-26T01:54:44.178343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.509587+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:56:12.509587+0000","last_peered":"2024-04-26T01:56:12.509587+0000","last_clean":"2024-04-26T01:56:12.509587+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:56:12.509587+0000","last_undegraded":"2024-04-26T01:56:12.509587+0000","last_fullsized":"2024-04-26T01:56:12.509587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:13.043482+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:56:13.043482+0000","last_peered":"2024-04-26T01:56:13.043482+0000","last_clean":"2024-04-26T01:56:13.043482+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:56:13.043482+0000","last_undegraded":"2024-04-26T01:56:13.043482+0000","last_fullsized":"2024-04-26T01:56:13.043482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.520762+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:56:12.520762+0000","last_peered":"2024-04-26T01:56:12.520762+0000","last_clean":"2024-04-26T01:56:12.520762+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:56:12.520762+0000","last_undegraded":"2024-04-26T01:56:12.520762+0000","last_fullsized":"2024-04-26T01:56:12.520762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175261+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:54:44.175261+0000","last_peered":"2024-04-26T01:54:44.175261+0000","last_clean":"2024-04-26T01:54:44.175261+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:54:44.175261+0000","last_undegraded":"2024-04-26T01:54:44.175261+0000","last_fullsized":"2024-04-26T01:54:44.175261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526655+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:56:12.526655+0000","last_peered":"2024-04-26T01:56:12.526655+0000","last_clean":"2024-04-26T01:56:12.526655+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:56:12.526655+0000","last_undegraded":"2024-04-26T01:56:12.526655+0000","last_fullsized":"2024-04-26T01:56:12.526655+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.190446+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:54:44.190446+0000","last_peered":"2024-04-26T01:54:44.190446+0000","last_clean":"2024-04-26T01:54:44.190446+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:54:44.190446+0000","last_undegraded":"2024-04-26T01:54:44.190446+0000","last_fullsized":"2024-04-26T01:54:44.190446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.491240+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:56:12.491240+0000","last_peered":"2024-04-26T01:56:12.491240+0000","last_clean":"2024-04-26T01:56:12.491240+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:56:12.491240+0000","last_undegraded":"2024-04-26T01:56:12.491240+0000","last_fullsized":"2024-04-26T01:56:12.491240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181150+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:54:44.181150+0000","last_peered":"2024-04-26T01:54:44.181150+0000","last_clean":"2024-04-26T01:54:44.181150+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:54:44.181150+0000","last_undegraded":"2024-04-26T01:54:44.181150+0000","last_fullsized":"2024-04-26T01:54:44.181150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505565+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:56:12.505565+0000","last_peered":"2024-04-26T01:56:12.505565+0000","last_clean":"2024-04-26T01:56:12.505565+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:56:12.505565+0000","last_undegraded":"2024-04-26T01:56:12.505565+0000","last_fullsized":"2024-04-26T01:56:12.505565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175037+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:54:44.175037+0000","last_peered":"2024-04-26T01:54:44.175037+0000","last_clean":"2024-04-26T01:54:44.175037+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:54:44.175037+0000","last_undegraded":"2024-04-26T01:54:44.175037+0000","last_fullsized":"2024-04-26T01:54:44.175037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526301+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:56:12.526301+0000","last_peered":"2024-04-26T01:56:12.526301+0000","last_clean":"2024-04-26T01:56:12.526301+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:56:12.526301+0000","last_undegraded":"2024-04-26T01:56:12.526301+0000","last_fullsized":"2024-04-26T01:56:12.526301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178004+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:54:44.178004+0000","last_peered":"2024-04-26T01:54:44.178004+0000","last_clean":"2024-04-26T01:54:44.178004+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:54:44.178004+0000","last_undegraded":"2024-04-26T01:54:44.178004+0000","last_fullsized":"2024-04-26T01:54:44.178004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521457+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:56:12.521457+0000","last_peered":"2024-04-26T01:56:12.521457+0000","last_clean":"2024-04-26T01:56:12.521457+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:56:12.521457+0000","last_undegraded":"2024-04-26T01:56:12.521457+0000","last_fullsized":"2024-04-26T01:56:12.521457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175297+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:54:44.175297+0000","last_peered":"2024-04-26T01:54:44.175297+0000","last_clean":"2024-04-26T01:54:44.175297+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:54:44.175297+0000","last_undegraded":"2024-04-26T01:54:44.175297+0000","last_fullsized":"2024-04-26T01:54:44.175297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521083+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:56:12.521083+0000","last_peered":"2024-04-26T01:56:12.521083+0000","last_clean":"2024-04-26T01:56:12.521083+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:56:12.521083+0000","last_undegraded":"2024-04-26T01:56:12.521083+0000","last_fullsized":"2024-04-26T01:56:12.521083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375063+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:54:44.375063+0000","last_peered":"2024-04-26T01:54:44.375063+0000","last_clean":"2024-04-26T01:54:44.375063+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:54:44.375063+0000","last_undegraded":"2024-04-26T01:54:44.375063+0000","last_fullsized":"2024-04-26T01:54:44.375063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.517609+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:56:12.517609+0000","last_peered":"2024-04-26T01:56:12.517609+0000","last_clean":"2024-04-26T01:56:12.517609+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:56:12.517609+0000","last_undegraded":"2024-04-26T01:56:12.517609+0000","last_fullsized":"2024-04-26T01:56:12.517609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377311+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:54:44.377311+0000","last_peered":"2024-04-26T01:54:44.377311+0000","last_clean":"2024-04-26T01:54:44.377311+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:54:44.377311+0000","last_undegraded":"2024-04-26T01:54:44.377311+0000","last_fullsized":"2024-04-26T01:54:44.377311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.514785+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:56:12.514785+0000","last_peered":"2024-04-26T01:56:12.514785+0000","last_clean":"2024-04-26T01:56:12.514785+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:56:12.514785+0000","last_undegraded":"2024-04-26T01:56:12.514785+0000","last_fullsized":"2024-04-26T01:56:12.514785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505585+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:56:12.505585+0000","last_peered":"2024-04-26T01:56:12.505585+0000","last_clean":"2024-04-26T01:56:12.505585+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:56:12.505585+0000","last_undegraded":"2024-04-26T01:56:12.505585+0000","last_fullsized":"2024-04-26T01:56:12.505585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.508552+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:56:12.508552+0000","last_peered":"2024-04-26T01:56:12.508552+0000","last_clean":"2024-04-26T01:56:12.508552+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:56:12.508552+0000","last_undegraded":"2024-04-26T01:56:12.508552+0000","last_fullsized":"2024-04-26T01:56:12.508552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529510+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:56:12.529510+0000","last_peered":"2024-04-26T01:56:12.529510+0000","last_clean":"2024-04-26T01:56:12.529510+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:56:12.529510+0000","last_undegraded":"2024-04-26T01:56:12.529510+0000","last_fullsized":"2024-04-26T01:56:12.529510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525989+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:56:12.525989+0000","last_peered":"2024-04-26T01:56:12.525989+0000","last_clean":"2024-04-26T01:56:12.525989+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:56:12.525989+0000","last_undegraded":"2024-04-26T01:56:12.525989+0000","last_fullsized":"2024-04-26T01:56:12.525989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524789+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:56:12.524789+0000","last_peered":"2024-04-26T01:56:12.524789+0000","last_clean":"2024-04-26T01:56:12.524789+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:56:12.524789+0000","last_undegraded":"2024-04-26T01:56:12.524789+0000","last_fullsized":"2024-04-26T01:56:12.524789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.504591+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:56:12.504591+0000","last_peered":"2024-04-26T01:56:12.504591+0000","last_clean":"2024-04-26T01:56:12.504591+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:56:12.504591+0000","last_undegraded":"2024-04-26T01:56:12.504591+0000","last_fullsized":"2024-04-26T01:56:12.504591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.516471+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:56:12.516471+0000","last_peered":"2024-04-26T01:56:12.516471+0000","last_clean":"2024-04-26T01:56:12.516471+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:56:12.516471+0000","last_undegraded":"2024-04-26T01:56:12.516471+0000","last_fullsized":"2024-04-26T01:56:12.516471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529573+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:56:12.529573+0000","last_peered":"2024-04-26T01:56:12.529573+0000","last_clean":"2024-04-26T01:56:12.529573+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:56:12.529573+0000","last_undegraded":"2024-04-26T01:56:12.529573+0000","last_fullsized":"2024-04-26T01:56:12.529573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525720+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:56:12.525720+0000","last_peered":"2024-04-26T01:56:12.525720+0000","last_clean":"2024-04-26T01:56:12.525720+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:56:12.525720+0000","last_undegraded":"2024-04-26T01:56:12.525720+0000","last_fullsized":"2024-04-26T01:56:12.525720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529940+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:56:12.529940+0000","last_peered":"2024-04-26T01:56:12.529940+0000","last_clean":"2024-04-26T01:56:12.529940+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:56:12.529940+0000","last_undegraded":"2024-04-26T01:56:12.529940+0000","last_fullsized":"2024-04-26T01:56:12.529940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526037+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:56:12.526037+0000","last_peered":"2024-04-26T01:56:12.526037+0000","last_clean":"2024-04-26T01:56:12.526037+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:56:12.526037+0000","last_undegraded":"2024-04-26T01:56:12.526037+0000","last_fullsized":"2024-04-26T01:56:12.526037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.527056+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:56:12.527056+0000","last_peered":"2024-04-26T01:56:12.527056+0000","last_clean":"2024-04-26T01:56:12.527056+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:56:12.527056+0000","last_undegraded":"2024-04-26T01:56:12.527056+0000","last_fullsized":"2024-04-26T01:56:12.527056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505923+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:56:12.505923+0000","last_peered":"2024-04-26T01:56:12.505923+0000","last_clean":"2024-04-26T01:56:12.505923+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:56:12.505923+0000","last_undegraded":"2024-04-26T01:56:12.505923+0000","last_fullsized":"2024-04-26T01:56:12.505923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63382,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":262,"num_read_kb":195,"num_write":88,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":190146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351025,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":482937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612661,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1136200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776185,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":483519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037824,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":483541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1074004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":483054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574951,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":545521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":483027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":545555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":483275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888151,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":482959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724626,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1073604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:56:35.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:56:35.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:34 smithi071 ceph-mon[32578]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:56:35.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:35.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:35.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:34 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:35.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:34 smithi045 ceph-mon[32520]: from='client.24838 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:56:35.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:56:35.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:56:35.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:34 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:56:35.587 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:36.073 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:35 smithi037 ceph-mon[27906]: pgmap v407: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:35 smithi071 ceph-mon[32578]: pgmap v407: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:35 smithi045 ceph-mon[32520]: pgmap v407: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:37.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:37 smithi037 ceph-mon[27906]: pgmap v408: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.970+0000 7fd9a0000700 1 -- 172.21.15.37:0/598227698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 msgr2=0x7fd99810dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.970+0000 7fd9a0000700 1 --2- 172.21.15.37:0/598227698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 0x7fd99810dcd0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd9840099c0 tx=0x7fd984009cd0 comp rx=0 tx=0).stop 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 -- 172.21.15.37:0/598227698 shutdown_connections 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 --2- 172.21.15.37:0/598227698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 --2- 172.21.15.37:0/598227698 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99810e3b0 0x7fd998115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 --2- 172.21.15.37:0/598227698 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 0x7fd99810dcd0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 -- 172.21.15.37:0/598227698 >> 172.21.15.37:0/598227698 conn(0x7fd998076210 msgr2=0x7fd998107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 -- 172.21.15.37:0/598227698 shutdown_connections 2024-04-26T01:56:37.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.971+0000 7fd9a0000700 1 -- 172.21.15.37:0/598227698 wait complete. 2024-04-26T01:56:37.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.972+0000 7fd9a0000700 1 Processor -- start 2024-04-26T01:56:37.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.972+0000 7fd9a0000700 1 -- start start 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.972+0000 7fd9a0000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981a5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.972+0000 7fd9a0000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 0x7fd9981a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd9a0000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99810e3b0 0x7fd99819f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd9a0000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd998119220 con 0x7fd9981092d0 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd9a0000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981190a0 con 0x7fd99810e3b0 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd9a0000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981193a0 con 0x7fd998109c80 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981a5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40596/0 (socket says 172.21.15.37:40596) 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 -- 172.21.15.37:0/3165679173 learned_addr learned my addr 172.21.15.37:0/3165679173 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:37.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99cc00700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 0x7fd9981a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:37.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 -- 172.21.15.37:0/3165679173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 msgr2=0x7fd9981a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:37.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 0x7fd9981a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:37.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 -- 172.21.15.37:0/3165679173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99810e3b0 msgr2=0x7fd99819f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:37.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99810e3b0 0x7fd99819f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.973+0000 7fd99d600700 1 -- 172.21.15.37:0/3165679173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99819fd40 con 0x7fd9981092d0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.974+0000 7fd99d600700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981a5420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00ef10 tx=0x7fd98c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.974+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c00ca70 con 0x7fd9981092d0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.974+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99819ffd0 con 0x7fd9981092d0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.974+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c019070 con 0x7fd9981092d0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.974+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c014480 con 0x7fd9981092d0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.974+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9981ac2f0 con 0x7fd9981092d0 2024-04-26T01:56:37.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.975+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd998069730 con 0x7fd9981092d0 2024-04-26T01:56:37.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.979+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd98c012050 con 0x7fd9981092d0 2024-04-26T01:56:37.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.981+0000 7fd996000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd9800a3c40 0x7fd9800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:37.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.981+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fd98c0d0b30 con 0x7fd9981092d0 2024-04-26T01:56:37.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.981+0000 7fd99cc00700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd9800a3c40 0x7fd9800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:37.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.982+0000 7fd99cc00700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd9800a3c40 0x7fd9800a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd984005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:37.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:37.984+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd98c010040 con 0x7fd9981092d0 2024-04-26T01:56:38.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:37 smithi071 ceph-mon[32578]: pgmap v408: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:38.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.210+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd998062e80 con 0x7fd9800a3c40 2024-04-26T01:56:38.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.221+0000 7fd996000700 1 -- 172.21.15.37:0/3165679173 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258751 (secure 0 0 0) 0x7fd998062e80 con 0x7fd9800a3c40 2024-04-26T01:56:38.224 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:38.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.227+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd9800a3c40 msgr2=0x7fd9800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.227+0000 7fd9a0000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd9800a3c40 0x7fd9800a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd984000c00 tx=0x7fd984005fb0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.227+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 msgr2=0x7fd9981a5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.227+0000 7fd9a0000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981a5420 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd98c00ef10 tx=0x7fd98c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 shutdown_connections 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9981092d0 0x7fd9981a5420 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fd9800a3c40 0x7fd9800a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd99810e3b0 0x7fd99819f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 --2- 172.21.15.37:0/3165679173 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd998109c80 0x7fd9981a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 >> 172.21.15.37:0/3165679173 conn(0x7fd998076210 msgr2=0x7fd998077e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 shutdown_connections 2024-04-26T01:56:38.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:38.228+0000 7fd9a0000700 1 -- 172.21.15.37:0/3165679173 wait complete. 2024-04-26T01:56:38.229 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:56:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:37 smithi045 ceph-mon[32520]: pgmap v408: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:38.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:38 smithi037 ceph-mon[27906]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:56:39.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:38 smithi071 ceph-mon[32578]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:56:39.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:38 smithi045 ceph-mon[32520]: from='client.14709 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:56:39.379 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-26T01:56:36.577659+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653984,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":398,"num_read_kb":314,"num_write":271,"num_write_kb":3030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":712,"ondisk_log_size":712,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371032,"kb_used_data":13560,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570792,"statfs":{"total":1151940427776,"available":1151560491008,"internally_reserved":0,"allocated":13885440,"data_stored":7757196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005753"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524517+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:56:12.524517+0000","last_peered":"2024-04-26T01:56:12.524517+0000","last_clean":"2024-04-26T01:56:12.524517+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:56:12.524517+0000","last_undegraded":"2024-04-26T01:56:12.524517+0000","last_fullsized":"2024-04-26T01:56:12.524517+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519684+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:56:12.519684+0000","last_peered":"2024-04-26T01:56:12.519684+0000","last_clean":"2024-04-26T01:56:12.519684+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:56:12.519684+0000","last_undegraded":"2024-04-26T01:56:12.519684+0000","last_fullsized":"2024-04-26T01:56:12.519684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528221+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:56:12.528221+0000","last_peered":"2024-04-26T01:56:12.528221+0000","last_clean":"2024-04-26T01:56:12.528221+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:56:12.528221+0000","last_undegraded":"2024-04-26T01:56:12.528221+0000","last_fullsized":"2024-04-26T01:56:12.528221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.504394+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:56:12.504394+0000","last_peered":"2024-04-26T01:56:12.504394+0000","last_clean":"2024-04-26T01:56:12.504394+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:56:12.504394+0000","last_undegraded":"2024-04-26T01:56:12.504394+0000","last_fullsized":"2024-04-26T01:56:12.504394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525240+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:56:12.525240+0000","last_peered":"2024-04-26T01:56:12.525240+0000","last_clean":"2024-04-26T01:56:12.525240+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:56:12.525240+0000","last_undegraded":"2024-04-26T01:56:12.525240+0000","last_fullsized":"2024-04-26T01:56:12.525240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.506176+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:56:12.506176+0000","last_peered":"2024-04-26T01:56:12.506176+0000","last_clean":"2024-04-26T01:56:12.506176+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:56:12.506176+0000","last_undegraded":"2024-04-26T01:56:12.506176+0000","last_fullsized":"2024-04-26T01:56:12.506176+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.502939+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:56:12.502939+0000","last_peered":"2024-04-26T01:56:12.502939+0000","last_clean":"2024-04-26T01:56:12.502939+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:56:12.502939+0000","last_undegraded":"2024-04-26T01:56:12.502939+0000","last_fullsized":"2024-04-26T01:56:12.502939+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528586+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:56:12.528586+0000","last_peered":"2024-04-26T01:56:12.528586+0000","last_clean":"2024-04-26T01:56:12.528586+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:56:12.528586+0000","last_undegraded":"2024-04-26T01:56:12.528586+0000","last_fullsized":"2024-04-26T01:56:12.528586+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.523595+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:56:12.523595+0000","last_peered":"2024-04-26T01:56:12.523595+0000","last_clean":"2024-04-26T01:56:12.523595+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:56:12.523595+0000","last_undegraded":"2024-04-26T01:56:12.523595+0000","last_fullsized":"2024-04-26T01:56:12.523595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.511960+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:56:12.511960+0000","last_peered":"2024-04-26T01:56:12.511960+0000","last_clean":"2024-04-26T01:56:12.511960+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:56:12.511960+0000","last_undegraded":"2024-04-26T01:56:12.511960+0000","last_fullsized":"2024-04-26T01:56:12.511960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.520329+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:56:12.520329+0000","last_peered":"2024-04-26T01:56:12.520329+0000","last_clean":"2024-04-26T01:56:12.520329+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:56:12.520329+0000","last_undegraded":"2024-04-26T01:56:12.520329+0000","last_fullsized":"2024-04-26T01:56:12.520329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.509394+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:56:12.509394+0000","last_peered":"2024-04-26T01:56:12.509394+0000","last_clean":"2024-04-26T01:56:12.509394+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:56:12.509394+0000","last_undegraded":"2024-04-26T01:56:12.509394+0000","last_fullsized":"2024-04-26T01:56:12.509394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.511929+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:56:12.511929+0000","last_peered":"2024-04-26T01:56:12.511929+0000","last_clean":"2024-04-26T01:56:12.511929+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:56:12.511929+0000","last_undegraded":"2024-04-26T01:56:12.511929+0000","last_fullsized":"2024-04-26T01:56:12.511929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521178+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:56:12.521178+0000","last_peered":"2024-04-26T01:56:12.521178+0000","last_clean":"2024-04-26T01:56:12.521178+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:56:12.521178+0000","last_undegraded":"2024-04-26T01:56:12.521178+0000","last_fullsized":"2024-04-26T01:56:12.521178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.522934+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:56:12.522934+0000","last_peered":"2024-04-26T01:56:12.522934+0000","last_clean":"2024-04-26T01:56:12.522934+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:56:12.522934+0000","last_undegraded":"2024-04-26T01:56:12.522934+0000","last_fullsized":"2024-04-26T01:56:12.522934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528575+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:56:12.528575+0000","last_peered":"2024-04-26T01:56:12.528575+0000","last_clean":"2024-04-26T01:56:12.528575+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:56:12.528575+0000","last_undegraded":"2024-04-26T01:56:12.528575+0000","last_fullsized":"2024-04-26T01:56:12.528575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524424+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:56:12.524424+0000","last_peered":"2024-04-26T01:56:12.524424+0000","last_clean":"2024-04-26T01:56:12.524424+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:56:12.524424+0000","last_undegraded":"2024-04-26T01:56:12.524424+0000","last_fullsized":"2024-04-26T01:56:12.524424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519455+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:56:12.519455+0000","last_peered":"2024-04-26T01:56:12.519455+0000","last_clean":"2024-04-26T01:56:12.519455+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:56:12.519455+0000","last_undegraded":"2024-04-26T01:56:12.519455+0000","last_fullsized":"2024-04-26T01:56:12.519455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529406+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:56:12.529406+0000","last_peered":"2024-04-26T01:56:12.529406+0000","last_clean":"2024-04-26T01:56:12.529406+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:56:12.529406+0000","last_undegraded":"2024-04-26T01:56:12.529406+0000","last_fullsized":"2024-04-26T01:56:12.529406+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.187845+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:54:44.187845+0000","last_peered":"2024-04-26T01:54:44.187845+0000","last_clean":"2024-04-26T01:54:44.187845+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:54:44.187845+0000","last_undegraded":"2024-04-26T01:54:44.187845+0000","last_fullsized":"2024-04-26T01:54:44.187845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529646+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:56:12.529646+0000","last_peered":"2024-04-26T01:56:12.529646+0000","last_clean":"2024-04-26T01:56:12.529646+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:56:12.529646+0000","last_undegraded":"2024-04-26T01:56:12.529646+0000","last_fullsized":"2024-04-26T01:56:12.529646+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.184671+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:54:44.184671+0000","last_peered":"2024-04-26T01:54:44.184671+0000","last_clean":"2024-04-26T01:54:44.184671+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:54:44.184671+0000","last_undegraded":"2024-04-26T01:54:44.184671+0000","last_fullsized":"2024-04-26T01:54:44.184671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528040+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:56:12.528040+0000","last_peered":"2024-04-26T01:56:12.528040+0000","last_clean":"2024-04-26T01:56:12.528040+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:56:12.528040+0000","last_undegraded":"2024-04-26T01:56:12.528040+0000","last_fullsized":"2024-04-26T01:56:12.528040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178505+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:54:44.178505+0000","last_peered":"2024-04-26T01:54:44.178505+0000","last_clean":"2024-04-26T01:54:44.178505+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:54:44.178505+0000","last_undegraded":"2024-04-26T01:54:44.178505+0000","last_fullsized":"2024-04-26T01:54:44.178505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521140+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:56:12.521140+0000","last_peered":"2024-04-26T01:56:12.521140+0000","last_clean":"2024-04-26T01:56:12.521140+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:56:12.521140+0000","last_undegraded":"2024-04-26T01:56:12.521140+0000","last_fullsized":"2024-04-26T01:56:12.521140+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178772+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:54:44.178772+0000","last_peered":"2024-04-26T01:54:44.178772+0000","last_clean":"2024-04-26T01:54:44.178772+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:54:44.178772+0000","last_undegraded":"2024-04-26T01:54:44.178772+0000","last_fullsized":"2024-04-26T01:54:44.178772+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.510658+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:56:12.510658+0000","last_peered":"2024-04-26T01:56:12.510658+0000","last_clean":"2024-04-26T01:56:12.510658+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:56:12.510658+0000","last_undegraded":"2024-04-26T01:56:12.510658+0000","last_fullsized":"2024-04-26T01:56:12.510658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375060+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:54:44.375060+0000","last_peered":"2024-04-26T01:54:44.375060+0000","last_clean":"2024-04-26T01:54:44.375060+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:54:44.375060+0000","last_undegraded":"2024-04-26T01:54:44.375060+0000","last_fullsized":"2024-04-26T01:54:44.375060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.502283+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:56:12.502283+0000","last_peered":"2024-04-26T01:56:12.502283+0000","last_clean":"2024-04-26T01:56:12.502283+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:56:12.502283+0000","last_undegraded":"2024-04-26T01:56:12.502283+0000","last_fullsized":"2024-04-26T01:56:12.502283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375951+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:54:44.375951+0000","last_peered":"2024-04-26T01:54:44.375951+0000","last_clean":"2024-04-26T01:54:44.375951+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:54:44.375951+0000","last_undegraded":"2024-04-26T01:54:44.375951+0000","last_fullsized":"2024-04-26T01:54:44.375951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525262+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:56:12.525262+0000","last_peered":"2024-04-26T01:56:12.525262+0000","last_clean":"2024-04-26T01:56:12.525262+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:56:12.525262+0000","last_undegraded":"2024-04-26T01:56:12.525262+0000","last_fullsized":"2024-04-26T01:56:12.525262+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376752+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:54:44.376752+0000","last_peered":"2024-04-26T01:54:44.376752+0000","last_clean":"2024-04-26T01:54:44.376752+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:54:44.376752+0000","last_undegraded":"2024-04-26T01:54:44.376752+0000","last_fullsized":"2024-04-26T01:54:44.376752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529432+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:56:12.529432+0000","last_peered":"2024-04-26T01:56:12.529432+0000","last_clean":"2024-04-26T01:56:12.529432+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:56:12.529432+0000","last_undegraded":"2024-04-26T01:56:12.529432+0000","last_fullsized":"2024-04-26T01:56:12.529432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375038+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:54:44.375038+0000","last_peered":"2024-04-26T01:54:44.375038+0000","last_clean":"2024-04-26T01:54:44.375038+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:54:44.375038+0000","last_undegraded":"2024-04-26T01:54:44.375038+0000","last_fullsized":"2024-04-26T01:54:44.375038+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529239+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:56:12.529239+0000","last_peered":"2024-04-26T01:56:12.529239+0000","last_clean":"2024-04-26T01:56:12.529239+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:56:12.529239+0000","last_undegraded":"2024-04-26T01:56:12.529239+0000","last_fullsized":"2024-04-26T01:56:12.529239+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376847+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:54:44.376847+0000","last_peered":"2024-04-26T01:54:44.376847+0000","last_clean":"2024-04-26T01:54:44.376847+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:54:44.376847+0000","last_undegraded":"2024-04-26T01:54:44.376847+0000","last_fullsized":"2024-04-26T01:54:44.376847+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.520527+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:56:12.520527+0000","last_peered":"2024-04-26T01:56:12.520527+0000","last_clean":"2024-04-26T01:56:12.520527+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:56:12.520527+0000","last_undegraded":"2024-04-26T01:56:12.520527+0000","last_fullsized":"2024-04-26T01:56:12.520527+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377265+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:54:44.377265+0000","last_peered":"2024-04-26T01:54:44.377265+0000","last_clean":"2024-04-26T01:54:44.377265+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:54:44.377265+0000","last_undegraded":"2024-04-26T01:54:44.377265+0000","last_fullsized":"2024-04-26T01:54:44.377265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.516160+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:56:12.516160+0000","last_peered":"2024-04-26T01:56:12.516160+0000","last_clean":"2024-04-26T01:56:12.516160+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:56:12.516160+0000","last_undegraded":"2024-04-26T01:56:12.516160+0000","last_fullsized":"2024-04-26T01:56:12.516160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375230+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:54:44.375230+0000","last_peered":"2024-04-26T01:54:44.375230+0000","last_clean":"2024-04-26T01:54:44.375230+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:54:44.375230+0000","last_undegraded":"2024-04-26T01:54:44.375230+0000","last_fullsized":"2024-04-26T01:54:44.375230+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524594+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:56:12.524594+0000","last_peered":"2024-04-26T01:56:12.524594+0000","last_clean":"2024-04-26T01:56:12.524594+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:56:12.524594+0000","last_undegraded":"2024-04-26T01:56:12.524594+0000","last_fullsized":"2024-04-26T01:56:12.524594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'31","reported_seq":314,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:31.720549+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:56:31.720549+0000","last_peered":"2024-04-26T01:56:31.720549+0000","last_clean":"2024-04-26T01:56:31.720549+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:56:31.720549+0000","last_undegraded":"2024-04-26T01:56:31.720549+0000","last_fullsized":"2024-04-26T01:56:31.720549+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62584,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":256,"num_read_kb":189,"num_write":31,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.512803+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:56:12.512803+0000","last_peered":"2024-04-26T01:56:12.512803+0000","last_clean":"2024-04-26T01:56:12.512803+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:56:12.512803+0000","last_undegraded":"2024-04-26T01:56:12.512803+0000","last_fullsized":"2024-04-26T01:56:12.512803+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175227+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:54:44.175227+0000","last_peered":"2024-04-26T01:54:44.175227+0000","last_clean":"2024-04-26T01:54:44.175227+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:54:44.175227+0000","last_undegraded":"2024-04-26T01:54:44.175227+0000","last_fullsized":"2024-04-26T01:54:44.175227+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529156+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:56:12.529156+0000","last_peered":"2024-04-26T01:56:12.529156+0000","last_clean":"2024-04-26T01:56:12.529156+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:56:12.529156+0000","last_undegraded":"2024-04-26T01:56:12.529156+0000","last_fullsized":"2024-04-26T01:56:12.529156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.374977+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:54:44.374977+0000","last_peered":"2024-04-26T01:54:44.374977+0000","last_clean":"2024-04-26T01:54:44.374977+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:54:44.374977+0000","last_undegraded":"2024-04-26T01:54:44.374977+0000","last_fullsized":"2024-04-26T01:54:44.374977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519960+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:56:12.519960+0000","last_peered":"2024-04-26T01:56:12.519960+0000","last_clean":"2024-04-26T01:56:12.519960+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:56:12.519960+0000","last_undegraded":"2024-04-26T01:56:12.519960+0000","last_fullsized":"2024-04-26T01:56:12.519960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.174994+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:54:44.174994+0000","last_peered":"2024-04-26T01:54:44.174994+0000","last_clean":"2024-04-26T01:54:44.174994+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:54:44.174994+0000","last_undegraded":"2024-04-26T01:54:44.174994+0000","last_fullsized":"2024-04-26T01:54:44.174994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.512438+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:56:12.512438+0000","last_peered":"2024-04-26T01:56:12.512438+0000","last_clean":"2024-04-26T01:56:12.512438+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:56:12.512438+0000","last_undegraded":"2024-04-26T01:56:12.512438+0000","last_fullsized":"2024-04-26T01:56:12.512438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.198548+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:54:44.198548+0000","last_peered":"2024-04-26T01:54:44.198548+0000","last_clean":"2024-04-26T01:54:44.198548+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:54:44.198548+0000","last_undegraded":"2024-04-26T01:54:44.198548+0000","last_fullsized":"2024-04-26T01:54:44.198548+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.522989+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:56:12.522989+0000","last_peered":"2024-04-26T01:56:12.522989+0000","last_clean":"2024-04-26T01:56:12.522989+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:56:12.522989+0000","last_undegraded":"2024-04-26T01:56:12.522989+0000","last_fullsized":"2024-04-26T01:56:12.522989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178306+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:54:44.178306+0000","last_peered":"2024-04-26T01:54:44.178306+0000","last_clean":"2024-04-26T01:54:44.178306+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:54:44.178306+0000","last_undegraded":"2024-04-26T01:54:44.178306+0000","last_fullsized":"2024-04-26T01:54:44.178306+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505709+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:56:12.505709+0000","last_peered":"2024-04-26T01:56:12.505709+0000","last_clean":"2024-04-26T01:56:12.505709+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:56:12.505709+0000","last_undegraded":"2024-04-26T01:56:12.505709+0000","last_fullsized":"2024-04-26T01:56:12.505709+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.182950+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:54:44.182950+0000","last_peered":"2024-04-26T01:54:44.182950+0000","last_clean":"2024-04-26T01:54:44.182950+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:54:44.182950+0000","last_undegraded":"2024-04-26T01:54:44.182950+0000","last_fullsized":"2024-04-26T01:54:44.182950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.500966+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:56:12.500966+0000","last_peered":"2024-04-26T01:56:12.500966+0000","last_clean":"2024-04-26T01:56:12.500966+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:56:12.500966+0000","last_undegraded":"2024-04-26T01:56:12.500966+0000","last_fullsized":"2024-04-26T01:56:12.500966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376577+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:54:44.376577+0000","last_peered":"2024-04-26T01:54:44.376577+0000","last_clean":"2024-04-26T01:54:44.376577+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:54:44.376577+0000","last_undegraded":"2024-04-26T01:54:44.376577+0000","last_fullsized":"2024-04-26T01:54:44.376577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.511810+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:56:12.511810+0000","last_peered":"2024-04-26T01:56:12.511810+0000","last_clean":"2024-04-26T01:56:12.511810+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:56:12.511810+0000","last_undegraded":"2024-04-26T01:56:12.511810+0000","last_fullsized":"2024-04-26T01:56:12.511810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.496771+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:56:12.496771+0000","last_peered":"2024-04-26T01:56:12.496771+0000","last_clean":"2024-04-26T01:56:12.496771+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:56:12.496771+0000","last_undegraded":"2024-04-26T01:56:12.496771+0000","last_fullsized":"2024-04-26T01:56:12.496771+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376367+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:54:44.376367+0000","last_peered":"2024-04-26T01:54:44.376367+0000","last_clean":"2024-04-26T01:54:44.376367+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:54:44.376367+0000","last_undegraded":"2024-04-26T01:54:44.376367+0000","last_fullsized":"2024-04-26T01:54:44.376367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528127+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:56:12.528127+0000","last_peered":"2024-04-26T01:56:12.528127+0000","last_clean":"2024-04-26T01:56:12.528127+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:56:12.528127+0000","last_undegraded":"2024-04-26T01:56:12.528127+0000","last_fullsized":"2024-04-26T01:56:12.528127+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175097+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:54:44.175097+0000","last_peered":"2024-04-26T01:54:44.175097+0000","last_clean":"2024-04-26T01:54:44.175097+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:54:44.175097+0000","last_undegraded":"2024-04-26T01:54:44.175097+0000","last_fullsized":"2024-04-26T01:54:44.175097+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376592+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:54:44.376592+0000","last_peered":"2024-04-26T01:54:44.376592+0000","last_clean":"2024-04-26T01:54:44.376592+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:54:44.376592+0000","last_undegraded":"2024-04-26T01:54:44.376592+0000","last_fullsized":"2024-04-26T01:54:44.376592+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505430+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:56:12.505430+0000","last_peered":"2024-04-26T01:56:12.505430+0000","last_clean":"2024-04-26T01:56:12.505430+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:56:12.505430+0000","last_undegraded":"2024-04-26T01:56:12.505430+0000","last_fullsized":"2024-04-26T01:56:12.505430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376779+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:54:44.376779+0000","last_peered":"2024-04-26T01:54:44.376779+0000","last_clean":"2024-04-26T01:54:44.376779+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:54:44.376779+0000","last_undegraded":"2024-04-26T01:54:44.376779+0000","last_fullsized":"2024-04-26T01:54:44.376779+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178343+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:54:44.178343+0000","last_peered":"2024-04-26T01:54:44.178343+0000","last_clean":"2024-04-26T01:54:44.178343+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:54:44.178343+0000","last_undegraded":"2024-04-26T01:54:44.178343+0000","last_fullsized":"2024-04-26T01:54:44.178343+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.509587+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:56:12.509587+0000","last_peered":"2024-04-26T01:56:12.509587+0000","last_clean":"2024-04-26T01:56:12.509587+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:56:12.509587+0000","last_undegraded":"2024-04-26T01:56:12.509587+0000","last_fullsized":"2024-04-26T01:56:12.509587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'10","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:13.043482+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:56:13.043482+0000","last_peered":"2024-04-26T01:56:13.043482+0000","last_clean":"2024-04-26T01:56:13.043482+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:56:13.043482+0000","last_undegraded":"2024-04-26T01:56:13.043482+0000","last_fullsized":"2024-04-26T01:56:13.043482+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.520762+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:56:12.520762+0000","last_peered":"2024-04-26T01:56:12.520762+0000","last_clean":"2024-04-26T01:56:12.520762+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:56:12.520762+0000","last_undegraded":"2024-04-26T01:56:12.520762+0000","last_fullsized":"2024-04-26T01:56:12.520762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175261+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:54:44.175261+0000","last_peered":"2024-04-26T01:54:44.175261+0000","last_clean":"2024-04-26T01:54:44.175261+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:54:44.175261+0000","last_undegraded":"2024-04-26T01:54:44.175261+0000","last_fullsized":"2024-04-26T01:54:44.175261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526655+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:56:12.526655+0000","last_peered":"2024-04-26T01:56:12.526655+0000","last_clean":"2024-04-26T01:56:12.526655+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:56:12.526655+0000","last_undegraded":"2024-04-26T01:56:12.526655+0000","last_fullsized":"2024-04-26T01:56:12.526655+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.190446+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:54:44.190446+0000","last_peered":"2024-04-26T01:54:44.190446+0000","last_clean":"2024-04-26T01:54:44.190446+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:54:44.190446+0000","last_undegraded":"2024-04-26T01:54:44.190446+0000","last_fullsized":"2024-04-26T01:54:44.190446+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.491240+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:56:12.491240+0000","last_peered":"2024-04-26T01:56:12.491240+0000","last_clean":"2024-04-26T01:56:12.491240+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:56:12.491240+0000","last_undegraded":"2024-04-26T01:56:12.491240+0000","last_fullsized":"2024-04-26T01:56:12.491240+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.181150+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:54:44.181150+0000","last_peered":"2024-04-26T01:54:44.181150+0000","last_clean":"2024-04-26T01:54:44.181150+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:54:44.181150+0000","last_undegraded":"2024-04-26T01:54:44.181150+0000","last_fullsized":"2024-04-26T01:54:44.181150+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505565+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:56:12.505565+0000","last_peered":"2024-04-26T01:56:12.505565+0000","last_clean":"2024-04-26T01:56:12.505565+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:56:12.505565+0000","last_undegraded":"2024-04-26T01:56:12.505565+0000","last_fullsized":"2024-04-26T01:56:12.505565+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175037+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:54:44.175037+0000","last_peered":"2024-04-26T01:54:44.175037+0000","last_clean":"2024-04-26T01:54:44.175037+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:54:44.175037+0000","last_undegraded":"2024-04-26T01:54:44.175037+0000","last_fullsized":"2024-04-26T01:54:44.175037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526301+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:56:12.526301+0000","last_peered":"2024-04-26T01:56:12.526301+0000","last_clean":"2024-04-26T01:56:12.526301+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:56:12.526301+0000","last_undegraded":"2024-04-26T01:56:12.526301+0000","last_fullsized":"2024-04-26T01:56:12.526301+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.178004+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:54:44.178004+0000","last_peered":"2024-04-26T01:54:44.178004+0000","last_clean":"2024-04-26T01:54:44.178004+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:54:44.178004+0000","last_undegraded":"2024-04-26T01:54:44.178004+0000","last_fullsized":"2024-04-26T01:54:44.178004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521457+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:56:12.521457+0000","last_peered":"2024-04-26T01:56:12.521457+0000","last_clean":"2024-04-26T01:56:12.521457+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:56:12.521457+0000","last_undegraded":"2024-04-26T01:56:12.521457+0000","last_fullsized":"2024-04-26T01:56:12.521457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.175297+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:54:44.175297+0000","last_peered":"2024-04-26T01:54:44.175297+0000","last_clean":"2024-04-26T01:54:44.175297+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:54:44.175297+0000","last_undegraded":"2024-04-26T01:54:44.175297+0000","last_fullsized":"2024-04-26T01:54:44.175297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521083+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:56:12.521083+0000","last_peered":"2024-04-26T01:56:12.521083+0000","last_clean":"2024-04-26T01:56:12.521083+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:56:12.521083+0000","last_undegraded":"2024-04-26T01:56:12.521083+0000","last_fullsized":"2024-04-26T01:56:12.521083+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'1","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.375063+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:54:44.375063+0000","last_peered":"2024-04-26T01:54:44.375063+0000","last_clean":"2024-04-26T01:54:44.375063+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:54:44.375063+0000","last_undegraded":"2024-04-26T01:54:44.375063+0000","last_fullsized":"2024-04-26T01:54:44.375063+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.517609+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:56:12.517609+0000","last_peered":"2024-04-26T01:56:12.517609+0000","last_clean":"2024-04-26T01:56:12.517609+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:56:12.517609+0000","last_undegraded":"2024-04-26T01:56:12.517609+0000","last_fullsized":"2024-04-26T01:56:12.517609+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377311+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:54:44.377311+0000","last_peered":"2024-04-26T01:54:44.377311+0000","last_clean":"2024-04-26T01:54:44.377311+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:54:44.377311+0000","last_undegraded":"2024-04-26T01:54:44.377311+0000","last_fullsized":"2024-04-26T01:54:44.377311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.514785+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:56:12.514785+0000","last_peered":"2024-04-26T01:56:12.514785+0000","last_clean":"2024-04-26T01:56:12.514785+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:56:12.514785+0000","last_undegraded":"2024-04-26T01:56:12.514785+0000","last_fullsized":"2024-04-26T01:56:12.514785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505585+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:56:12.505585+0000","last_peered":"2024-04-26T01:56:12.505585+0000","last_clean":"2024-04-26T01:56:12.505585+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:56:12.505585+0000","last_undegraded":"2024-04-26T01:56:12.505585+0000","last_fullsized":"2024-04-26T01:56:12.505585+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.508552+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:56:12.508552+0000","last_peered":"2024-04-26T01:56:12.508552+0000","last_clean":"2024-04-26T01:56:12.508552+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:56:12.508552+0000","last_undegraded":"2024-04-26T01:56:12.508552+0000","last_fullsized":"2024-04-26T01:56:12.508552+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529510+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:56:12.529510+0000","last_peered":"2024-04-26T01:56:12.529510+0000","last_clean":"2024-04-26T01:56:12.529510+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:56:12.529510+0000","last_undegraded":"2024-04-26T01:56:12.529510+0000","last_fullsized":"2024-04-26T01:56:12.529510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525989+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:56:12.525989+0000","last_peered":"2024-04-26T01:56:12.525989+0000","last_clean":"2024-04-26T01:56:12.525989+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:56:12.525989+0000","last_undegraded":"2024-04-26T01:56:12.525989+0000","last_fullsized":"2024-04-26T01:56:12.525989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524789+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:56:12.524789+0000","last_peered":"2024-04-26T01:56:12.524789+0000","last_clean":"2024-04-26T01:56:12.524789+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:56:12.524789+0000","last_undegraded":"2024-04-26T01:56:12.524789+0000","last_fullsized":"2024-04-26T01:56:12.524789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.504591+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:56:12.504591+0000","last_peered":"2024-04-26T01:56:12.504591+0000","last_clean":"2024-04-26T01:56:12.504591+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:56:12.504591+0000","last_undegraded":"2024-04-26T01:56:12.504591+0000","last_fullsized":"2024-04-26T01:56:12.504591+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.516471+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:56:12.516471+0000","last_peered":"2024-04-26T01:56:12.516471+0000","last_clean":"2024-04-26T01:56:12.516471+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:56:12.516471+0000","last_undegraded":"2024-04-26T01:56:12.516471+0000","last_fullsized":"2024-04-26T01:56:12.516471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529573+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:56:12.529573+0000","last_peered":"2024-04-26T01:56:12.529573+0000","last_clean":"2024-04-26T01:56:12.529573+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:56:12.529573+0000","last_undegraded":"2024-04-26T01:56:12.529573+0000","last_fullsized":"2024-04-26T01:56:12.529573+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525720+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:56:12.525720+0000","last_peered":"2024-04-26T01:56:12.525720+0000","last_clean":"2024-04-26T01:56:12.525720+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:56:12.525720+0000","last_undegraded":"2024-04-26T01:56:12.525720+0000","last_fullsized":"2024-04-26T01:56:12.525720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529940+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:56:12.529940+0000","last_peered":"2024-04-26T01:56:12.529940+0000","last_clean":"2024-04-26T01:56:12.529940+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:56:12.529940+0000","last_undegraded":"2024-04-26T01:56:12.529940+0000","last_fullsized":"2024-04-26T01:56:12.529940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526037+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:56:12.526037+0000","last_peered":"2024-04-26T01:56:12.526037+0000","last_clean":"2024-04-26T01:56:12.526037+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:56:12.526037+0000","last_undegraded":"2024-04-26T01:56:12.526037+0000","last_fullsized":"2024-04-26T01:56:12.526037+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.527056+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:56:12.527056+0000","last_peered":"2024-04-26T01:56:12.527056+0000","last_clean":"2024-04-26T01:56:12.527056+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:56:12.527056+0000","last_undegraded":"2024-04-26T01:56:12.527056+0000","last_fullsized":"2024-04-26T01:56:12.527056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505923+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:56:12.505923+0000","last_peered":"2024-04-26T01:56:12.505923+0000","last_clean":"2024-04-26T01:56:12.505923+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:56:12.505923+0000","last_undegraded":"2024-04-26T01:56:12.505923+0000","last_fullsized":"2024-04-26T01:56:12.505923+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63382,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":262,"num_read_kb":195,"num_write":88,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":282624,"data_stored":190146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":73,"ondisk_log_size":73,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":983040,"data_stored":482937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612661,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28644,"kb_used_data":1628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716508,"statfs":{"total":95995035648,"available":95965704192,"internally_reserved":0,"allocated":1667072,"data_stored":1136200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776186,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":483519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037825,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":991232,"data_stored":483541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299462,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1074004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":483054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":545521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":483027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36372,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708780,"statfs":{"total":95995035648,"available":95957790720,"internally_reserved":0,"allocated":1060864,"data_stored":545555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":483275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888151,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":482959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724627,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28572,"kb_used_data":1556,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716580,"statfs":{"total":95995035648,"available":95965777920,"internally_reserved":0,"allocated":1593344,"data_stored":1073604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":62584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":62778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:56:39.383 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:56:39.383 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T01:56:39.383 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T01:56:39.383 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T01:56:39.910 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:40.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:39 smithi037 ceph-mon[27906]: pgmap v409: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:40.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:39 smithi071 ceph-mon[32578]: pgmap v409: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:40.155 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:39 smithi045 ceph-mon[32520]: pgmap v409: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:42.054 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:41 smithi037 ceph-mon[27906]: pgmap v410: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:42.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:41 smithi071 ceph-mon[32578]: pgmap v410: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:42.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.226+0000 7f428b800700 1 -- 172.21.15.37:0/1692538480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 msgr2=0x7f4284075050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:42.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.226+0000 7f428b800700 1 --2- 172.21.15.37:0/1692538480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 0x7f4284075050 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f4278009990 tx=0x7f4278009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 -- 172.21.15.37:0/1692538480 shutdown_connections 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 --2- 172.21.15.37:0/1692538480 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f428410f4a0 0x7f4284111890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 --2- 172.21.15.37:0/1692538480 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 0x7f4284075050 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 --2- 172.21.15.37:0/1692538480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4284075590 0x7f428410ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 -- 172.21.15.37:0/1692538480 >> 172.21.15.37:0/1692538480 conn(0x7f4284079ba0 msgr2=0x7f4284079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 -- 172.21.15.37:0/1692538480 shutdown_connections 2024-04-26T01:56:42.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.227+0000 7f428b800700 1 -- 172.21.15.37:0/1692538480 wait complete. 2024-04-26T01:56:42.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.228+0000 7f428b800700 1 Processor -- start 2024-04-26T01:56:42.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.228+0000 7f428b800700 1 -- start start 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f428b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 0x7f428419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f428b800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4284075590 0x7f428419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f428b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 0x7f428419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f428b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4284114e90 con 0x7f428410f4a0 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f428b800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4284114d10 con 0x7f4284075590 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f428b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4284115010 con 0x7f4284074c70 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f4289800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 0x7f428419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:42.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f4289800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 0x7f428419fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34456/0 (socket says 172.21.15.37:34456) 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f4289800700 1 -- 172.21.15.37:0/4191541436 learned_addr learned my addr 172.21.15.37:0/4191541436 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f4288e00700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 0x7f428419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.229+0000 7f4283e00700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4284075590 0x7f428419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.230+0000 7f4289800700 1 -- 172.21.15.37:0/4191541436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 msgr2=0x7f428419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.230+0000 7f4289800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 0x7f428419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.230+0000 7f4289800700 1 -- 172.21.15.37:0/4191541436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4284075590 msgr2=0x7f428419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.230+0000 7f4289800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4284075590 0x7f428419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.230+0000 7f4289800700 1 -- 172.21.15.37:0/4191541436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42841a02f0 con 0x7f428410f4a0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.230+0000 7f4289800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 0x7f428419fbb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f427c00efc0 tx=0x7f427c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.231+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0092e0 con 0x7f428410f4a0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.231+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f427c004bb0 con 0x7f428410f4a0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.231+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c004020 con 0x7f428410f4a0 2024-04-26T01:56:42.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.231+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42841a0580 con 0x7f428410f4a0 2024-04-26T01:56:42.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.231+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42841a7e60 con 0x7f428410f4a0 2024-04-26T01:56:42.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.232+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4284069730 con 0x7f428410f4a0 2024-04-26T01:56:42.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.235+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f427c01b030 con 0x7f428410f4a0 2024-04-26T01:56:42.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.237+0000 7f4281600700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f426c0a3c40 0x7f426c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:42.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.237+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f427c0cfdd0 con 0x7f428410f4a0 2024-04-26T01:56:42.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.237+0000 7f4288e00700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f426c0a3c40 0x7f426c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:42.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.238+0000 7f4288e00700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f426c0a3c40 0x7f426c0a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f427800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:42.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.242+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f427c022020 con 0x7f428410f4a0 2024-04-26T01:56:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:41 smithi045 ceph-mon[32520]: pgmap v410: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:42.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.527+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f42841a8030 con 0x7f428410f4a0 2024-04-26T01:56:42.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.527+0000 7f4281600700 1 -- 172.21.15.37:0/4191541436 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f427c010040 con 0x7f428410f4a0 2024-04-26T01:56:42.527 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:42.527 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T01:56:42.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f426c0a3c40 msgr2=0x7f426c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:42.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f426c0a3c40 0x7f426c0a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4278006010 tx=0x7f427800b260 comp rx=0 tx=0).stop 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 msgr2=0x7f428419fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 0x7f428419fbb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f427c00efc0 tx=0x7f427c00c3a0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 shutdown_connections 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f428410f4a0 0x7f428419fbb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f426c0a3c40 0x7f426c0a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4284074c70 0x7f428419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 --2- 172.21.15.37:0/4191541436 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4284075590 0x7f428419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 >> 172.21.15.37:0/4191541436 conn(0x7f4284079ba0 msgr2=0x7f42840ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 shutdown_connections 2024-04-26T01:56:42.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:42.531+0000 7f428b800700 1 -- 172.21.15.37:0/4191541436 wait complete. 2024-04-26T01:56:43.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:42 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4191541436' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:56:43.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:42 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4191541436' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:56:43.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:42 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4191541436' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T01:56:43.529 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T01:56:43.529 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:56:44.012 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:44.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:43 smithi071 ceph-mon[32578]: pgmap v411: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:44.161 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:43 smithi037 ceph-mon[27906]: pgmap v411: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:43 smithi045 ceph-mon[32520]: pgmap v411: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:46.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.086+0000 7f9537c00700 1 -- 172.21.15.37:0/2444209979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9530077890 msgr2=0x7f95301133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.086+0000 7f9537c00700 1 --2- 172.21.15.37:0/2444209979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9530077890 0x7f95301133b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f951c009990 tx=0x7f951c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.086 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:45 smithi037 ceph-mon[27906]: pgmap v412: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.086+0000 7f9537c00700 1 -- 172.21.15.37:0/2444209979 shutdown_connections 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.086+0000 7f9537c00700 1 --2- 172.21.15.37:0/2444209979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f95301138f0 0x7f9530115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.087+0000 7f9537c00700 1 --2- 172.21.15.37:0/2444209979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530076f70 0x7f9530077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.087+0000 7f9537c00700 1 --2- 172.21.15.37:0/2444209979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9530077890 0x7f95301133b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.087+0000 7f9537c00700 1 -- 172.21.15.37:0/2444209979 >> 172.21.15.37:0/2444209979 conn(0x7f9530100740 msgr2=0x7f9530102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.087+0000 7f9537c00700 1 -- 172.21.15.37:0/2444209979 shutdown_connections 2024-04-26T01:56:46.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.087+0000 7f9537c00700 1 -- 172.21.15.37:0/2444209979 wait complete. 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.088+0000 7f9537c00700 1 Processor -- start 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.088+0000 7f9537c00700 1 -- start start 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9537c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9530076f70 0x7f95301a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9537c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530077890 0x7f95301a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9537c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 0x7f953019f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9537c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95301192d0 con 0x7f95301138f0 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9537c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9530119150 con 0x7f9530077890 2024-04-26T01:56:46.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9537c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9530119450 con 0x7f9530076f70 2024-04-26T01:56:46.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9535c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 0x7f953019f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9535c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 0x7f953019f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34466/0 (socket says 172.21.15.37:34466) 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9535c00700 1 -- 172.21.15.37:0/2445032919 learned_addr learned my addr 172.21.15.37:0/2445032919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f9535200700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9530076f70 0x7f95301a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.089+0000 7f952fe00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530077890 0x7f95301a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535200700 1 -- 172.21.15.37:0/2445032919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530077890 msgr2=0x7f95301a5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535200700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530077890 0x7f95301a5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535200700 1 -- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 msgr2=0x7f953019f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535200700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 0x7f953019f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535200700 1 -- 172.21.15.37:0/2445032919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f953019fd30 con 0x7f9530076f70 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 0x7f953019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f952fe00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530077890 0x7f95301a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9535200700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9530076f70 0x7f95301a5410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f952400c9c0 con 0x7f9530076f70 2024-04-26T01:56:46.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f952400cb60 con 0x7f9530076f70 2024-04-26T01:56:46.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95301a0020 con 0x7f9530076f70 2024-04-26T01:56:46.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.090+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9524018680 con 0x7f9530076f70 2024-04-26T01:56:46.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.091+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95301ac2e0 con 0x7f9530076f70 2024-04-26T01:56:46.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.092+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9530069730 con 0x7f9530076f70 2024-04-26T01:56:46.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.094+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9524010430 con 0x7f9530076f70 2024-04-26T01:56:46.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.095+0000 7f952d600700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95180a3cc0 0x7f95180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:46.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.095+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f95240d0530 con 0x7f9530076f70 2024-04-26T01:56:46.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.096+0000 7f952fe00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95180a3cc0 0x7f95180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:46.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.096+0000 7f952fe00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95180a3cc0 0x7f95180a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9530103e00 tx=0x7f951c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:46.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.100+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f952409cec0 con 0x7f9530076f70 2024-04-26T01:56:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:45 smithi071 ceph-mon[32578]: pgmap v412: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:45 smithi045 ceph-mon[32520]: pgmap v412: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:46.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.376+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f953002df80 con 0x7f9530076f70 2024-04-26T01:56:46.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.377+0000 7f952d600700 1 -- 172.21.15.37:0/2445032919 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f952409c7c0 con 0x7f9530076f70 2024-04-26T01:56:46.376 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:46.376 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95180a3cc0 msgr2=0x7f95180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95180a3cc0 0x7f95180a6180 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9530103e00 tx=0x7f951c00b260 comp rx=0 tx=0).stop 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9530076f70 msgr2=0x7f95301a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9530076f70 0x7f95301a5410 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f952400e9f0 tx=0x7f952400ed00 comp rx=0 tx=0).stop 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 shutdown_connections 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95301138f0 0x7f953019f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f95180a3cc0 0x7f95180a6180 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9530076f70 0x7f95301a5410 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 --2- 172.21.15.37:0/2445032919 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9530077890 0x7f95301a5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:46.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 >> 172.21.15.37:0/2445032919 conn(0x7f9530100740 msgr2=0x7f9530102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:46.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.380+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 shutdown_connections 2024-04-26T01:56:46.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:46.381+0000 7f9537c00700 1 -- 172.21.15.37:0/2445032919 wait complete. 2024-04-26T01:56:46.380 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:56:47.096 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:46 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2445032919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:47.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:46 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2445032919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:47.171 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:56:47.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:46 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2445032919' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:47.660 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:48.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:47 smithi071 ceph-mon[32578]: pgmap v413: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:48.152 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:47 smithi037 ceph-mon[27906]: pgmap v413: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:47 smithi045 ceph-mon[32520]: pgmap v413: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 -- 172.21.15.37:0/2145242696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4104ee0 msgr2=0x7f1ae41052c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2145242696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4104ee0 0x7f1ae41052c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1ad400b0d0 tx=0x7f1ad400b3e0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 -- 172.21.15.37:0/2145242696 shutdown_connections 2024-04-26T01:56:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2145242696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae4109fc0 0x7f1ae4111b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2145242696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4104ee0 0x7f1ae41052c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2145242696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4105890 0x7f1ae41098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 -- 172.21.15.37:0/2145242696 >> 172.21.15.37:0/2145242696 conn(0x7f1ae4100770 msgr2=0x7f1ae4102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 -- 172.21.15.37:0/2145242696 shutdown_connections 2024-04-26T01:56:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.095+0000 7f1aeb000700 1 -- 172.21.15.37:0/2145242696 wait complete. 2024-04-26T01:56:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.096+0000 7f1aeb000700 1 Processor -- start 2024-04-26T01:56:50.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 -- start start 2024-04-26T01:56:50.096 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:49 smithi037 ceph-mon[27906]: pgmap v414: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4104ee0 0x7f1ae419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 0x7f1ae419b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae4109fc0 0x7f1ae419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4114eb0 con 0x7f1ae4109fc0 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4114d30 con 0x7f1ae4104ee0 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.097+0000 7f1aeb000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae4115030 con 0x7f1ae4105890 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1adb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 0x7f1ae419b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4104ee0 0x7f1ae419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae9000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae4109fc0 0x7f1ae419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1adb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 0x7f1ae419b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43134/0 (socket says 172.21.15.37:43134) 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1adb400700 1 -- 172.21.15.37:0/2118301664 learned_addr learned my addr 172.21.15.37:0/2118301664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae3e00700 1 -- 172.21.15.37:0/2118301664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 msgr2=0x7f1ae419b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae3e00700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 0x7f1ae419b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae3e00700 1 -- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae4109fc0 msgr2=0x7f1ae419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae3e00700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae4109fc0 0x7f1ae419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1ae3e00700 1 -- 172.21.15.37:0/2118301664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae41a0360 con 0x7f1ae4104ee0 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.098+0000 7f1adb400700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 0x7f1ae419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:50.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.099+0000 7f1ae3e00700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4104ee0 0x7f1ae419b3a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4000f80 tx=0x7f1ad4007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.099+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad401f020 con 0x7f1ae4104ee0 2024-04-26T01:56:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.099+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ad4003c00 con 0x7f1ae4104ee0 2024-04-26T01:56:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.099+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad401b430 con 0x7f1ae4104ee0 2024-04-26T01:56:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.099+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae41a05f0 con 0x7f1ae4104ee0 2024-04-26T01:56:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.099+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae41a7ed0 con 0x7f1ae4104ee0 2024-04-26T01:56:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.101+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae41063c0 con 0x7f1ae4104ee0 2024-04-26T01:56:50.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.102+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1ad4022020 con 0x7f1ae4104ee0 2024-04-26T01:56:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.104+0000 7f1ae1600700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1ac80a3c40 0x7f1ac80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.104+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f1ad400f020 con 0x7f1ae4104ee0 2024-04-26T01:56:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.104+0000 7f1adb400700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1ac80a3c40 0x7f1ac80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.105+0000 7f1adb400700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1ac80a3c40 0x7f1ac80a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4100c20 tx=0x7f1acc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.108+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1ad409d960 con 0x7f1ae4104ee0 2024-04-26T01:56:50.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:49 smithi071 ceph-mon[32578]: pgmap v414: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:50.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:49 smithi045 ceph-mon[32520]: pgmap v414: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:50.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.389+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ae4069730 con 0x7f1ae4104ee0 2024-04-26T01:56:50.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.390+0000 7f1ae1600700 1 -- 172.21.15.37:0/2118301664 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f1ad409d260 con 0x7f1ae4104ee0 2024-04-26T01:56:50.389 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:50.389 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:56:50.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1ac80a3c40 msgr2=0x7f1ac80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1ac80a3c40 0x7f1ac80a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4100c20 tx=0x7f1acc009260 comp rx=0 tx=0).stop 2024-04-26T01:56:50.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4104ee0 msgr2=0x7f1ae419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:50.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4104ee0 0x7f1ae419b3a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4000f80 tx=0x7f1ad4007a00 comp rx=0 tx=0).stop 2024-04-26T01:56:50.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 shutdown_connections 2024-04-26T01:56:50.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ae4109fc0 0x7f1ae419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f1ac80a3c40 0x7f1ac80a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1ae4104ee0 0x7f1ae419b3a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 --2- 172.21.15.37:0/2118301664 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f1ae4105890 0x7f1ae419b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:50.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 >> 172.21.15.37:0/2118301664 conn(0x7f1ae4100770 msgr2=0x7f1ae4102af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:50.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 shutdown_connections 2024-04-26T01:56:50.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:50.394+0000 7f1aeb000700 1 -- 172.21.15.37:0/2118301664 wait complete. 2024-04-26T01:56:50.395 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:56:51.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:50 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2118301664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:51.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:50 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2118301664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:51.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:50 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2118301664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:51.432 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:56:51.909 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:52.128 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:51 smithi037 ceph-mon[27906]: pgmap v415: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:52.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:51 smithi071 ceph-mon[32578]: pgmap v415: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:52.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:51 smithi045 ceph-mon[32520]: pgmap v415: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:53.978 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:53 smithi037 ceph-mon[27906]: pgmap v416: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:53 smithi071 ceph-mon[32578]: pgmap v416: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:53 smithi045 ceph-mon[32520]: pgmap v416: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:56:54.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 -- 172.21.15.37:0/2098314670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90841034b0 msgr2=0x7f908410f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 --2- 172.21.15.37:0/2098314670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90841034b0 0x7f908410f920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f907c00b0d0 tx=0x7f907c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 -- 172.21.15.37:0/2098314670 shutdown_connections 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 --2- 172.21.15.37:0/2098314670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f90841034b0 0x7f908410f920 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 --2- 172.21.15.37:0/2098314670 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 0x7f9084107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 --2- 172.21.15.37:0/2098314670 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9084102af0 0x7f9084102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 -- 172.21.15.37:0/2098314670 >> 172.21.15.37:0/2098314670 conn(0x7f9084079ba0 msgr2=0x7f9084079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 -- 172.21.15.37:0/2098314670 shutdown_connections 2024-04-26T01:56:54.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.280+0000 7f9089400700 1 -- 172.21.15.37:0/2098314670 wait complete. 2024-04-26T01:56:54.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.282+0000 7f9089400700 1 Processor -- start 2024-04-26T01:56:54.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.282+0000 7f9089400700 1 -- start start 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.282+0000 7f9089400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9084102af0 0x7f908419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9089400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 0x7f908419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9089400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 0x7f908419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9089400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9084112ca0 con 0x7f9084102af0 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9089400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9084112b20 con 0x7f9084107940 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9089400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9084112e20 con 0x7f90841034b0 2024-04-26T01:56:54.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9082800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9084102af0 0x7f908419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 0x7f908419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 0x7f908419b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43156/0 (socket says 172.21.15.37:43156) 2024-04-26T01:56:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 -- 172.21.15.37:0/3499657775 learned_addr learned my addr 172.21.15.37:0/3499657775 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:54.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9083200700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 0x7f908419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 -- 172.21.15.37:0/3499657775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 msgr2=0x7f908419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 0x7f908419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 -- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9084102af0 msgr2=0x7f908419b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9084102af0 0x7f908419b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9079e00700 1 -- 172.21.15.37:0/3499657775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90841a02f0 con 0x7f90841034b0 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9082800700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9084102af0 0x7f908419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.283+0000 7f9083200700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 0x7f908419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:54.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.284+0000 7f9079e00700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 0x7f908419b820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9c0 tx=0x7f906c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.284+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c00cbd0 con 0x7f90841034b0 2024-04-26T01:56:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.284+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f906c00ee60 con 0x7f90841034b0 2024-04-26T01:56:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.284+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90841a05e0 con 0x7f90841034b0 2024-04-26T01:56:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.284+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906c0055c0 con 0x7f90841034b0 2024-04-26T01:56:54.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.284+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90841a7e60 con 0x7f90841034b0 2024-04-26T01:56:54.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.285+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9084069730 con 0x7f90841034b0 2024-04-26T01:56:54.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.287+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f906c004750 con 0x7f90841034b0 2024-04-26T01:56:54.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.289+0000 7f907be00700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90680a3c40 0x7f90680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:54.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.289+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f906c0d0700 con 0x7f90841034b0 2024-04-26T01:56:54.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.289+0000 7f9082800700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90680a3c40 0x7f90680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:54.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.290+0000 7f9082800700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90680a3c40 0x7f90680a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9074009680 tx=0x7f9074009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:54.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.293+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f906c09d090 con 0x7f90841034b0 2024-04-26T01:56:54.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.525+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f90841a8030 con 0x7f90841034b0 2024-04-26T01:56:54.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.527+0000 7f907be00700 1 -- 172.21.15.37:0/3499657775 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22499 (secure 0 0 0) 0x7f906c09c990 con 0x7f90841034b0 2024-04-26T01:56:54.526 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:54.526 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:44.157091+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:56:54.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90680a3c40 msgr2=0x7f90680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90680a3c40 0x7f90680a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9074009680 tx=0x7f9074009260 comp rx=0 tx=0).stop 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 msgr2=0x7f908419b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 0x7f908419b820 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f906c00e9c0 tx=0x7f906c00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 shutdown_connections 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9084102af0 0x7f908419b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90680a3c40 0x7f90680a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9084107940 0x7f908419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 --2- 172.21.15.37:0/3499657775 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f90841034b0 0x7f908419b820 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.530+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 >> 172.21.15.37:0/3499657775 conn(0x7f9084079ba0 msgr2=0x7f9084100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.531+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 shutdown_connections 2024-04-26T01:56:54.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:54.531+0000 7f9089400700 1 -- 172.21.15.37:0/3499657775 wait complete. 2024-04-26T01:56:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:54 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3499657775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:56:55.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:54 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3499657775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:56:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:54 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3499657775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:56:55.480 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:56:55.968 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:56:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:55 smithi071 ceph-mon[32578]: pgmap v417: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:56.192 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:55 smithi037 ceph-mon[27906]: pgmap v417: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:55 smithi045 ceph-mon[32520]: pgmap v417: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:58.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:57 smithi037 ceph-mon[27906]: pgmap v418: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:57 smithi071 ceph-mon[32578]: pgmap v418: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:57 smithi045 ceph-mon[32520]: pgmap v418: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:56:58.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 -- 172.21.15.37:0/27091450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc109e90 msgr2=0x7fa5cc111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/27091450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc109e90 0x7fa5cc111a60 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400b0d0 tx=0x7fa5c400b3e0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 -- 172.21.15.37:0/27091450 shutdown_connections 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/27091450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc109e90 0x7fa5cc111a60 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/27091450 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5cc104db0 0x7fa5cc105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/27091450 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc105760 0x7fa5cc1097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 -- 172.21.15.37:0/27091450 >> 172.21.15.37:0/27091450 conn(0x7fa5cc100680 msgr2=0x7fa5cc102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 -- 172.21.15.37:0/27091450 shutdown_connections 2024-04-26T01:56:58.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.315+0000 7fa5d1a00700 1 -- 172.21.15.37:0/27091450 wait complete. 2024-04-26T01:56:58.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.316+0000 7fa5d1a00700 1 Processor -- start 2024-04-26T01:56:58.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.316+0000 7fa5d1a00700 1 -- start start 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 0x7fa5cc19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5cc105760 0x7fa5cc19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc109e90 0x7fa5cc19fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1a00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc114d90 con 0x7fa5cc104db0 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1a00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc114c10 con 0x7fa5cc109e90 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1a00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa5cc114f10 con 0x7fa5cc105760 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 0x7fa5cc19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 0x7fa5cc19b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35414/0 (socket says 172.21.15.37:35414) 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 -- 172.21.15.37:0/2207514458 learned_addr learned my addr 172.21.15.37:0/2207514458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5c3e00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5cc105760 0x7fa5cc19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5d1000700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc109e90 0x7fa5cc19fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 -- 172.21.15.37:0/2207514458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5cc105760 msgr2=0x7fa5cc19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5cc105760 0x7fa5cc19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 -- 172.21.15.37:0/2207514458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc109e90 msgr2=0x7fa5cc19fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.317+0000 7fa5cbe00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc109e90 0x7fa5cc19fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5cbe00700 1 -- 172.21.15.37:0/2207514458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5cc1a0200 con 0x7fa5cc104db0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5d1000700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc109e90 0x7fa5cc19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5cbe00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 0x7fa5cc19b240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9c0 tx=0x7fa5bc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc00cbd0 con 0x7fa5cc104db0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5bc00ee60 con 0x7fa5cc104db0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc0055c0 con 0x7fa5cc104db0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5cc1a04f0 con 0x7fa5cc104db0 2024-04-26T01:56:58.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.318+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5cc1a7d70 con 0x7fa5cc104db0 2024-04-26T01:56:58.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.319+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5cc069730 con 0x7fa5cc104db0 2024-04-26T01:56:58.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.322+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa5bc004750 con 0x7fa5cc104db0 2024-04-26T01:56:58.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.324+0000 7fa5c9600700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa5b00a3c40 0x7fa5b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:56:58.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.324+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fa5bc0d0700 con 0x7fa5cc104db0 2024-04-26T01:56:58.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.324+0000 7fa5c3e00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa5b00a3c40 0x7fa5b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:56:58.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.325+0000 7fa5c3e00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa5b00a3c40 0x7fa5b00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009680 tx=0x7fa5b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:56:58.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.329+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa5bc09d090 con 0x7fa5cc104db0 2024-04-26T01:56:58.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.614+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5cc19c550 con 0x7fa5cc104db0 2024-04-26T01:56:58.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.616+0000 7fa5c9600700 1 -- 172.21.15.37:0/2207514458 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fa5bc09c990 con 0x7fa5cc104db0 2024-04-26T01:56:58.615 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:56:58.615 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:56:58.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.619+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa5b00a3c40 msgr2=0x7fa5b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.619+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa5b00a3c40 0x7fa5b00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4009680 tx=0x7fa5b4009260 comp rx=0 tx=0).stop 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.619+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 msgr2=0x7fa5cc19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.619+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 0x7fa5cc19b240 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00e9c0 tx=0x7fa5bc00ecd0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.619+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 shutdown_connections 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa5cc104db0 0x7fa5cc19b240 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fa5b00a3c40 0x7fa5b00a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa5cc109e90 0x7fa5cc19fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 --2- 172.21.15.37:0/2207514458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa5cc105760 0x7fa5cc19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:56:58.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 >> 172.21.15.37:0/2207514458 conn(0x7fa5cc100680 msgr2=0x7fa5cc101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:56:58.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 shutdown_connections 2024-04-26T01:56:58.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:56:58.620+0000 7fa5d1a00700 1 -- 172.21.15.37:0/2207514458 wait complete. 2024-04-26T01:56:58.620 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:56:59.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:58 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2207514458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:59.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:58 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2207514458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:59.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:58 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2207514458' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:56:59.613 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:54:12.162501+0000', 'modified': '2024-04-26T01:54:33.402956+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/60713297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 60713297}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 60713297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.71:6839/1148187051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1148187051}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1148187051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24514, 'qdb_cluster': [24514]} 2024-04-26T01:56:59.614 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T01:56:59.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:57:00.097 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:00.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:56:59 smithi071 ceph-mon[32578]: pgmap v419: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:00.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:56:59 smithi037 ceph-mon[27906]: pgmap v419: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:56:59 smithi045 ceph-mon[32520]: pgmap v419: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:02.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:01 smithi071 ceph-mon[32578]: pgmap v420: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:01 smithi037 ceph-mon[27906]: pgmap v420: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:01 smithi045 ceph-mon[32520]: pgmap v420: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 -- 172.21.15.37:0/1553041609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc06bd10 msgr2=0x7f5afc105560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 --2- 172.21.15.37:0/1553041609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc06bd10 0x7f5afc105560 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5af400b0a0 tx=0x7f5af400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 -- 172.21.15.37:0/1553041609 shutdown_connections 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 --2- 172.21.15.37:0/1553041609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc06bd10 0x7f5afc105560 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 --2- 172.21.15.37:0/1553041609 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 0x7f5afc06b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 --2- 172.21.15.37:0/1553041609 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5afc107e10 0x7f5afc06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 -- 172.21.15.37:0/1553041609 >> 172.21.15.37:0/1553041609 conn(0x7f5afc0786a0 msgr2=0x7f5afc078ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:02.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 -- 172.21.15.37:0/1553041609 shutdown_connections 2024-04-26T01:57:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.413+0000 7f5b02400700 1 -- 172.21.15.37:0/1553041609 wait complete. 2024-04-26T01:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.414+0000 7f5b02400700 1 Processor -- start 2024-04-26T01:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.414+0000 7f5b02400700 1 -- start start 2024-04-26T01:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5b02400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 0x7f5afc19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5b02400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5afc107e10 0x7f5afc19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5b02400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc1a4010 0x7f5afc1a4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5b02400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc119270 con 0x7f5afc1a4010 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5b02400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1190f0 con 0x7f5afc107e10 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5b02400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f5afc1193f0 con 0x7f5afc1079a0 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc1a4010 0x7f5afc1a4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5afb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 0x7f5afc19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc1a4010 0x7f5afc1a4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49312/0 (socket says 172.21.15.37:49312) 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5afbe00700 1 -- 172.21.15.37:0/786932051 learned_addr learned my addr 172.21.15.37:0/786932051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.415+0000 7f5af2a00700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5afc107e10 0x7f5afc19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5afb400700 1 -- 172.21.15.37:0/786932051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5afc107e10 msgr2=0x7f5afc19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5afb400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5afc107e10 0x7f5afc19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5afb400700 1 -- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc1a4010 msgr2=0x7f5afc1a4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5afb400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc1a4010 0x7f5afc1a4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5afb400700 1 -- 172.21.15.37:0/786932051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5afc1a4b30 con 0x7f5afc1079a0 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5afb400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 0x7f5afc19f740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00e9f0 tx=0x7f5aec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:02.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5af8c00700 1 -- 172.21.15.37:0/786932051 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec004220 con 0x7f5afc1079a0 2024-04-26T01:57:02.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5af8c00700 1 -- 172.21.15.37:0/786932051 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec0043c0 con 0x7f5afc1079a0 2024-04-26T01:57:02.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5af8c00700 1 -- 172.21.15.37:0/786932051 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec005020 con 0x7f5afc1079a0 2024-04-26T01:57:02.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.416+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5afc1a4e20 con 0x7f5afc1079a0 2024-04-26T01:57:02.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.417+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5afc1ac5b0 con 0x7f5afc1079a0 2024-04-26T01:57:02.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.420+0000 7f5af8c00700 1 -- 172.21.15.37:0/786932051 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5aec004750 con 0x7f5afc1079a0 2024-04-26T01:57:02.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.422+0000 7f5af8c00700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5ae00a3cc0 0x7f5ae00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.422+0000 7f5b0ce36b80 1 Processor -- start 2024-04-26T01:57:02.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5af8c00700 1 -- 172.21.15.37:0/786932051 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f5aec0d0300 con 0x7f5afc1079a0 2024-04-26T01:57:02.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 -- start start 2024-04-26T01:57:02.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5af2a00700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5ae00a3cc0 0x7f5ae00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 0x55b46ed326d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed32c10 0x55b46ed3d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3dab0 0x55b46ed3fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b46ec85e40 con 0x55b46ed32c10 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b46ec2c6e0 con 0x55b46ed322f0 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.423+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b46ec81e00 con 0x55b46ed3dab0 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5af2a00700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5ae00a3cc0 0x7f5ae00a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009680 tx=0x7f5ae4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5af2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed32c10 0x55b46ed3d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 0x55b46ed326d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3dab0 0x55b46ed3fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3dab0 0x55b46ed3fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54302/0 (socket says 172.21.15.37:54302) 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afbe00700 1 -- 172.21.15.37:0/3189892501 learned_addr learned my addr 172.21.15.37:0/3189892501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 0x55b46ed326d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:58652/0 (socket says 172.21.15.37:58652) 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 -- 172.21.15.37:0/3189892501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3dab0 msgr2=0x55b46ed3fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3dab0 0x55b46ed3fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 -- 172.21.15.37:0/3189892501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed32c10 msgr2=0x55b46ed3d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed32c10 0x55b46ed3d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 -- 172.21.15.37:0/3189892501 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46ec22630 con 0x55b46ed322f0 2024-04-26T01:57:02.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.424+0000 7f5afb400700 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 0x55b46ed326d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5aec09c290 tx=0x7f5aec09bf70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8f52ee24de0f46b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.425+0000 7f5af0c00700 1 -- 172.21.15.37:0/3189892501 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01c040 con 0x55b46ed322f0 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.425+0000 7f5af0c00700 1 -- 172.21.15.37:0/3189892501 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec0b5070 con 0x55b46ed322f0 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.425+0000 7f5af0c00700 1 -- 172.21.15.37:0/3189892501 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec09a970 con 0x55b46ed322f0 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/3189892501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 msgr2=0x55b46ed326d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 0x55b46ed326d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5aec09c290 tx=0x7f5aec09bf70 comp rx=0 tx=0).stop 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/3189892501 shutdown_connections 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3dab0 0x55b46ed3fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed322f0 0x55b46ed326d0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f5aec09c290 tx=0x7f5aec09bf70 comp rx=0 tx=0).stop 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/3189892501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed32c10 0x55b46ed3d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/3189892501 >> 172.21.15.37:0/3189892501 conn(0x55b46ec61550 msgr2=0x55b46ed2e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:02.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/3189892501 shutdown_connections 2024-04-26T01:57:02.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.426+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/3189892501 wait complete. 2024-04-26T01:57:02.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.427+0000 7f5b0ce36b80 1 -- start start 2024-04-26T01:57:02.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.427+0000 7f5b0ce36b80 1 Processor -- start 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed32c10 0x55b46ed3d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed3a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed3ad00 0x55b46ed3b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed405a0 con 0x55b46ed3dab0 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b46ec3f810 con 0x55b46ed3ad00 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed40720 con 0x55b46ed32c10 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5b0ce36b80 10 client.0 _ll_register_callbacks cb 0x55b46ec1e6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5af2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed3a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed32c10 0x55b46ed3d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5af2a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed3a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49330/0 (socket says 172.21.15.37:49330) 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5af2a00700 1 -- 172.21.15.37:0/1543631401 learned_addr learned my addr 172.21.15.37:0/1543631401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afbe00700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed3ad00 0x55b46ed3b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afb400700 1 -- 172.21.15.37:0/1543631401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed3ad00 msgr2=0x55b46ed3b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afb400700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed3ad00 0x55b46ed3b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afb400700 1 -- 172.21.15.37:0/1543631401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 msgr2=0x55b46ed3a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afb400700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed3a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5afb400700 1 -- 172.21.15.37:0/1543631401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46ebd3880 con 0x55b46ed32c10 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.428+0000 7f5af2a00700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed3a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5afb400700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed32c10 0x55b46ed3d470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c5b0 tx=0x7f5aec09c6d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27ece3c9c30aa0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:02.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5ae9600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5ae9600700 1 -- 172.21.15.37:0/1543631401 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01c030 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5ae9600700 1 -- 172.21.15.37:0/1543631401 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f5aec0a4d70 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aec0b5070 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5ae9600700 1 -- 172.21.15.37:0/1543631401 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5aec0a5200 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5b0ce36b80 10 client.24886 fetch_fsmap learned FSMap version 19 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b46ec1caa0 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.429+0000 7f5ae9600700 1 -- 172.21.15.37:0/1543631401 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aec0b5070 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5ae9600700 1 -- 172.21.15.37:0/1543631401 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5aec0bb1e0 con 0x55b46ed32c10 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5b0ce36b80 10 client.24886 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:57:02.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5b0ce36b80 10 client.24886 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5b0ce36b80 20 client.24886 populate_metadata read hostname 'smithi037' 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46ed4c600 0x55b46ed4e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5b0ce36b80 4 client.24886 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b46ec93630 con 0x55b46ed4c600 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.430+0000 7f5af2a00700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46ed4c600 0x55b46ed4e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.431+0000 7f5af2a00700 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46ed4c600 0x55b46ed4e7e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9e12db355a409 server_cookie=6f4caeb269919182 in_seq=0 out_seq=0 2024-04-26T01:57:02.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.431+0000 7f5ae9600700 10 client.24886 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5ae9600700 1 -- 172.21.15.37:0/1543631401 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b46ec93630 con 0x55b46ed4c600 2024-04-26T01:57:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5ae9600700 10 client.24886 handle_command_reply: tid=0 2024-04-26T01:57:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 client.24886 shutdown 2024-04-26T01:57:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed32c10 msgr2=0x55b46ed3d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed32c10 0x55b46ed3d470 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00c5b0 tx=0x7f5aec09c6d0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 shutdown_connections 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46ed4c600 0x55b46ed4e7e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed3ad00 0x55b46ed3b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed32c10 0x55b46ed3d470 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1543631401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed3a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5ae9600700 0 client.24886 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 >> 172.21.15.37:0/1543631401 conn(0x55b46ec61550 msgr2=0x55b46ed34c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 shutdown_connections 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.432+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1543631401 wait complete. 2024-04-26T01:57:02.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.433+0000 7f5b0ce36b80 20 client.24886 trim_cache size 0 max 16384 2024-04-26T01:57:02.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.455+0000 7f5b0ce36b80 1 Processor -- start 2024-04-26T01:57:02.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.455+0000 7f5b0ce36b80 1 -- start start 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed3c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed57c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed58180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed4cb20 con 0x55b46ed3dab0 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed4d1f0 con 0x55b46ed32c10 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed4d8c0 con 0x55b46ed3ad00 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed58180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed58180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49346/0 (socket says 172.21.15.37:49346) 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5afbe00700 1 -- 172.21.15.37:0/1795639007 learned_addr learned my addr 172.21.15.37:0/1795639007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5af2a00700 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed57c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.456+0000 7f5afb400700 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed3c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afbe00700 1 -- 172.21.15.37:0/1795639007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 msgr2=0x55b46ed57c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afbe00700 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed57c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afbe00700 1 -- 172.21.15.37:0/1795639007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 msgr2=0x55b46ed3c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afbe00700 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed3c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afbe00700 1 -- 172.21.15.37:0/1795639007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b46ed26020 con 0x55b46ed3dab0 2024-04-26T01:57:02.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afb400700 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed3c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:02.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5afbe00700 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed58180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5af400d010 tx=0x7f5af4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cb269209760b2d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:02.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.457+0000 7f5aebe00700 1 -- 172.21.15.37:0/1795639007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af400bbb0 con 0x55b46ed3dab0 2024-04-26T01:57:02.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5aebe00700 1 -- 172.21.15.37:0/1795639007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af400bd50 con 0x55b46ed3dab0 2024-04-26T01:57:02.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5aebe00700 1 -- 172.21.15.37:0/1795639007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af4007d50 con 0x55b46ed3dab0 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1795639007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 msgr2=0x55b46ed58180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed58180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5af400d010 tx=0x7f5af4009580 comp rx=0 tx=0).stop 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1795639007 shutdown_connections 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed58180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed3c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/1795639007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed57c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.458+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1795639007 >> 172.21.15.37:0/1795639007 conn(0x55b46ec61550 msgr2=0x55b46ed35dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.459+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1795639007 shutdown_connections 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.459+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/1795639007 wait complete. 2024-04-26T01:57:02.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.459+0000 7f5b0ce36b80 1 -- start start 2024-04-26T01:57:02.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.459+0000 7f5b0ce36b80 1 Processor -- start 2024-04-26T01:57:02.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed37ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed4e660 con 0x55b46ed3dab0 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed4df90 con 0x55b46ed32c10 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b46ed4ed30 con 0x55b46ed3ad00 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5b0ce36b80 10 client.0 _ll_register_callbacks cb 0x55b46ec1e6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed37570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:58670/0 (socket says 172.21.15.37:58670) 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed37ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 -- 172.21.15.37:0/4072858641 learned_addr learned my addr 172.21.15.37:0/4072858641 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5af2a00700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 -- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 msgr2=0x55b46ed37ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed37ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 -- 172.21.15.37:0/4072858641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 msgr2=0x55b46ed37ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed37ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afb400700 1 -- 172.21.15.37:0/4072858641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b46ed56010 con 0x55b46ed32c10 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.460+0000 7f5afbe00700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed37ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5afb400700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed37570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0097e0 tx=0x7f5aec00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c284effa414e912 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5ae9600700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5ae9600700 1 -- 172.21.15.37:0/4072858641 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aec01c070 con 0x55b46ed32c10 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5af400bd50 con 0x55b46ed32c10 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5ae9600700 1 -- 172.21.15.37:0/4072858641 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f5aec09b9e0 con 0x55b46ed32c10 2024-04-26T01:57:02.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5ae9600700 1 -- 172.21.15.37:0/4072858641 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5aec0a0940 con 0x55b46ed32c10 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5b0ce36b80 10 client.24851 fetch_fsmap learned FSMap version 19 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.461+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b46ed56010 con 0x55b46ed32c10 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5ae9600700 1 -- 172.21.15.37:0/4072858641 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5af400bd50 con 0x55b46ed32c10 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5ae9600700 1 -- 172.21.15.37:0/4072858641 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5aec0ac020 con 0x55b46ed32c10 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5b0ce36b80 10 client.24851 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5b0ce36b80 10 client.24851 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5b0ce36b80 20 client.24851 populate_metadata read hostname 'smithi037' 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46edb8f70 0x55b46edb7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5b0ce36b80 4 client.24851 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.462+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "status"}) -- 0x55b46ec8d6d0 con 0x55b46edb8f70 2024-04-26T01:57:02.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5af2a00700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46edb8f70 0x55b46edb7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:02.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5af2a00700 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46edb8f70 0x55b46edb7c80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174693d1580b4896 server_cookie=74ddec5f581cf23a in_seq=0 out_seq=0 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5ae9600700 10 client.24851 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5ae9600700 1 -- 172.21.15.37:0/4072858641 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55b46ec8d6d0 con 0x55b46edb8f70 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5ae9600700 10 client.24851 handle_command_reply: tid=0 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5b0ce36b80 1 client.24851 shutdown 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 msgr2=0x55b46ed37570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.463+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed37570 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5aec0097e0 tx=0x7f5aec00c8d0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 shutdown_connections 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b46edb8f70 0x55b46edb7c80 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b46ed3dab0 0x55b46ed37ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b46ed32c10 0x55b46ed37570 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5ae9600700 0 client.24851 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 --2- 172.21.15.37:0/4072858641 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b46ed3ad00 0x55b46ed37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 >> 172.21.15.37:0/4072858641 conn(0x55b46ec61550 msgr2=0x55b46ed3c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 shutdown_connections 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 1 -- 172.21.15.37:0/4072858641 wait complete. 2024-04-26T01:57:02.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.464+0000 7f5b0ce36b80 20 client.24851 trim_cache size 0 max 16384 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24514, 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 169.44337926599999, 2024-04-26T01:57:02.466 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 19, 2024-04-26T01:57:02.467 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-26T01:57:02.467 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T01:57:02.467 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 206.565652 2024-04-26T01:57:02.467 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T01:57:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5ae00a3cc0 msgr2=0x7f5ae00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5ae00a3cc0 0x7f5ae00a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f5ae4009680 tx=0x7f5ae4009260 comp rx=0 tx=0).stop 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 msgr2=0x7f5afc19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 0x7f5afc19f740 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5aec00e9f0 tx=0x7f5aec00ed00 comp rx=0 tx=0).stop 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 shutdown_connections 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5afc1a4010 0x7f5afc1a4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f5ae00a3cc0 0x7f5ae00a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f5afc1079a0 0x7f5afc19f740 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 --2- 172.21.15.37:0/786932051 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f5afc107e10 0x7f5afc19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 >> 172.21.15.37:0/786932051 conn(0x7f5afc0786a0 msgr2=0x7f5afc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 shutdown_connections 2024-04-26T01:57:02.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:02.465+0000 7f5b02400700 1 -- 172.21.15.37:0/786932051 wait complete. 2024-04-26T01:57:03.323 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T01:57:03.324 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T01:57:03.336 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T01:57:03.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T01:57:03.791 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:04.159 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:03 smithi037 ceph-mon[27906]: pgmap v421: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:03 smithi045 ceph-mon[32520]: pgmap v421: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:03 smithi071 ceph-mon[32578]: pgmap v421: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:06.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:05 smithi037 ceph-mon[27906]: pgmap v422: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 -- 172.21.15.37:0/340517650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a01018d0 msgr2=0x7f40a0101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 --2- 172.21.15.37:0/340517650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a01018d0 0x7f40a0101cf0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f40900098f0 tx=0x7f4090009c00 comp rx=0 tx=0).stop 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 -- 172.21.15.37:0/340517650 shutdown_connections 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 --2- 172.21.15.37:0/340517650 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0103470 0x7f40a0109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 --2- 172.21.15.37:0/340517650 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a01018d0 0x7f40a0101cf0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 --2- 172.21.15.37:0/340517650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a0102ad0 0x7f40a0102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 -- 172.21.15.37:0/340517650 >> 172.21.15.37:0/340517650 conn(0x7f40a00fce70 msgr2=0x7f40a00ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 -- 172.21.15.37:0/340517650 shutdown_connections 2024-04-26T01:57:06.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.122+0000 7f409fe00700 1 -- 172.21.15.37:0/340517650 wait complete. 2024-04-26T01:57:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.123+0000 7f409fe00700 1 Processor -- start 2024-04-26T01:57:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 -- start start 2024-04-26T01:57:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 0x7f40a006bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:06.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0102ad0 0x7f40a019d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0103470 0x7f40a019fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40a010d360 con 0x7f40a01018d0 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f40a010d1e0 con 0x7f40a0102ad0 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409fe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f40a010d4e0 con 0x7f40a0103470 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 0x7f40a006bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0103470 0x7f40a019fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 0x7f40a006bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49370/0 (socket says 172.21.15.37:49370) 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409ea00700 1 -- 172.21.15.37:0/135014547 learned_addr learned my addr 172.21.15.37:0/135014547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:06.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.124+0000 7f409e000700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0102ad0 0x7f40a019d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409ea00700 1 -- 172.21.15.37:0/135014547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0103470 msgr2=0x7f40a019fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409ea00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0103470 0x7f40a019fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409ea00700 1 -- 172.21.15.37:0/135014547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0102ad0 msgr2=0x7f40a019d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409ea00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0102ad0 0x7f40a019d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409ea00700 1 -- 172.21.15.37:0/135014547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a01a0790 con 0x7f40a01018d0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409e000700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0102ad0 0x7f40a019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.125+0000 7f409ea00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 0x7f40a006bb70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4090009810 tx=0x7f4090005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.126+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090005460 con 0x7f40a01018d0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.126+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a01a0a20 con 0x7f40a01018d0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.126+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4090005600 con 0x7f40a01018d0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.126+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4090003b80 con 0x7f40a01018d0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.126+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a01a0e70 con 0x7f40a01018d0 2024-04-26T01:57:06.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.127+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a0069730 con 0x7f40a01018d0 2024-04-26T01:57:06.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.129+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f409000b560 con 0x7f40a01018d0 2024-04-26T01:57:06.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.130+0000 7f4097400700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40840a3c40 0x7f40840a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:06.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.131+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f40900d1490 con 0x7f40a01018d0 2024-04-26T01:57:06.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.131+0000 7f409e000700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40840a3c40 0x7f40840a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:06.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.132+0000 7f409e000700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40840a3c40 0x7f40840a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f408c009750 tx=0x7f408c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:06.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.136+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4090013030 con 0x7f40a01018d0 2024-04-26T01:57:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:05 smithi045 ceph-mon[32520]: pgmap v422: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:06.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.389+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f40a0062e80 con 0x7f40840a3c40 2024-04-26T01:57:06.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.392+0000 7f4097400700 1 -- 172.21.15.37:0/135014547 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f40a0062e80 con 0x7f40840a3c40 2024-04-26T01:57:06.391 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-04-26T01:57:06.391 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-04-26T01:57:06.391 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-04-26T01:57:06.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.395+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40840a3c40 msgr2=0x7f40840a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:06.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.395+0000 7f409fe00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40840a3c40 0x7f40840a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f408c009750 tx=0x7f408c009310 comp rx=0 tx=0).stop 2024-04-26T01:57:06.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.395+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 msgr2=0x7f40a006bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:06.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.395+0000 7f409fe00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 0x7f40a006bb70 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4090009810 tx=0x7f4090005270 comp rx=0 tx=0).stop 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 shutdown_connections 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40a01018d0 0x7f40a006bb70 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f40840a3c40 0x7f40840a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f40a0103470 0x7f40a019fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 --2- 172.21.15.37:0/135014547 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f40a0102ad0 0x7f40a019d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 >> 172.21.15.37:0/135014547 conn(0x7f40a00fce70 msgr2=0x7f40a0105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 shutdown_connections 2024-04-26T01:57:06.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:06.396+0000 7f409fe00700 1 -- 172.21.15.37:0/135014547 wait complete. 2024-04-26T01:57:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:05 smithi071 ceph-mon[32578]: pgmap v422: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:07.234 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T01:57:07.725 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:07 smithi037 ceph-mon[27906]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:07 smithi037 ceph-mon[27906]: pgmap v423: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:07 smithi045 ceph-mon[32520]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:07 smithi045 ceph-mon[32520]: pgmap v423: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:07 smithi071 ceph-mon[32578]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:08.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:07 smithi071 ceph-mon[32578]: pgmap v423: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:10.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:09 smithi037 ceph-mon[27906]: pgmap v424: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.086+0000 7febc8e00700 1 -- 172.21.15.37:0/2027110589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4103540 msgr2=0x7febc4109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.086+0000 7febc8e00700 1 --2- 172.21.15.37:0/2027110589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4103540 0x7febc4109df0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7febbc00b0a0 tx=0x7febbc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 -- 172.21.15.37:0/2027110589 shutdown_connections 2024-04-26T01:57:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 --2- 172.21.15.37:0/2027110589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc41019a0 0x7febc4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 --2- 172.21.15.37:0/2027110589 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc4103540 0x7febc4109df0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 --2- 172.21.15.37:0/2027110589 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 0x7febc4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 -- 172.21.15.37:0/2027110589 >> 172.21.15.37:0/2027110589 conn(0x7febc40fcf40 msgr2=0x7febc40ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 -- 172.21.15.37:0/2027110589 shutdown_connections 2024-04-26T01:57:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.087+0000 7febc8e00700 1 -- 172.21.15.37:0/2027110589 wait complete. 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 Processor -- start 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 -- start start 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc41019a0 0x7febc419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 0x7febc419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 0x7febc41a23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febc410d420 con 0x7febc4103540 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7febc410d2a0 con 0x7febc4102ba0 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.088+0000 7febc8e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7febc410d5a0 con 0x7febc41019a0 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 0x7febc41a23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 0x7febc419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc2800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 0x7febc41a23a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39250/0 (socket says 172.21.15.37:39250) 2024-04-26T01:57:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc2800700 1 -- 172.21.15.37:0/2989788470 learned_addr learned my addr 172.21.15.37:0/2989788470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc41019a0 0x7febc419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1400700 1 -- 172.21.15.37:0/2989788470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc41019a0 msgr2=0x7febc419ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1400700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc41019a0 0x7febc419ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1400700 1 -- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 msgr2=0x7febc41a23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1400700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 0x7febc41a23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc1400700 1 -- 172.21.15.37:0/2989788470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febc41a2a40 con 0x7febc4102ba0 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.089+0000 7febc2800700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 0x7febc41a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.090+0000 7febc1400700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 0x7febc419b320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7febb000e9f0 tx=0x7febb000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.090+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb000c9c0 con 0x7febc4102ba0 2024-04-26T01:57:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.090+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febc41a2d30 con 0x7febc4102ba0 2024-04-26T01:57:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.090+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febb000cb60 con 0x7febc4102ba0 2024-04-26T01:57:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.090+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febb0018680 con 0x7febc4102ba0 2024-04-26T01:57:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.090+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febc41a3210 con 0x7febc4102ba0 2024-04-26T01:57:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.091+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febc4069730 con 0x7febc4102ba0 2024-04-26T01:57:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.093+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7febb0010430 con 0x7febc4102ba0 2024-04-26T01:57:10.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.094+0000 7febbaa00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7feba80a3c40 0x7feba80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:10.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.095+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7febb00d05b0 con 0x7febc4102ba0 2024-04-26T01:57:10.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.095+0000 7febc1e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7feba80a3c40 0x7feba80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:10.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.095+0000 7febc1e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7feba80a3c40 0x7feba80a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febc4102a00 tx=0x7febb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:10.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.099+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7febb009cf40 con 0x7febc4102ba0 2024-04-26T01:57:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:09 smithi045 ceph-mon[32520]: pgmap v424: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:10.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.342+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7febc40ff130 con 0x7feba80a3c40 2024-04-26T01:57:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.383+0000 7febbaa00700 1 -- 172.21.15.37:0/2989788470 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7febc40ff130 con 0x7feba80a3c40 2024-04-26T01:57:10.382 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (3m) 36s ago 3m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 23296243e7ec 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi045 running (3m) 64s ago 3m 19.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f1ba4643264a 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi071 running (3m) 64s ago 3m 15.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2f4077c0725f 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (3m) 36s ago 3m 13.2M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5884dcd8bed9 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi045 running (3m) 64s ago 3m 16.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 61f22fd4072f 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi071 running (3m) 64s ago 3m 14.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 6d15fa8b32d8 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (3m) 36s ago 3m 13.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8f3b1f24b5a6 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi045 running (3m) 64s ago 3m 16.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a9f2e6a17d7b 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi071 running (3m) 64s ago 3m 25.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3520e2016e1b 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (3m) 36s ago 3m 13.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 93f54a6229f2 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi045 running (3m) 64s ago 3m 14.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 a866d6b8464c 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi071 running (4m) 64s ago 4m 14.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 b8e9d2b16da6 2024-04-26T01:57:10.383 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (12m) 36s ago 12m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8472511c7553 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi045 *:8443,8765 running (10m) 64s ago 10m 452M - 19.0.0-3156-gd69f13d8 da4b004c09e8 62f7b75aa845 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi071 *:8443,8765 running (10m) 64s ago 10m 453M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3f8112940c8a 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (12m) 36s ago 12m 50.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 d2b78331dbde 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi045 running (10m) 64s ago 10m 41.5M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 f99f16de2f2b 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi071 running (10m) 64s ago 10m 42.0M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 b141b1cd90a2 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (9m) 36s ago 9m 49.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 191eaf470605 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi045 running (9m) 64s ago 9m 47.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 dac5211d3b1a 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi071 running (8m) 64s ago 8m 58.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 e92327bef026 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (8m) 36s ago 8m 54.3M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 cb8124ef9eaf 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi045 running (7m) 64s ago 7m 46.9M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 6c0096b13420 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi071 running (7m) 64s ago 7m 49.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 80a7b09db79d 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (6m) 36s ago 6m 51.4M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 62d3a9a28627 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi045 running (6m) 64s ago 6m 49.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 63f882b6abf3 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi071 running (5m) 64s ago 5m 48.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 ae247e50bb2e 2024-04-26T01:57:10.384 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (5m) 36s ago 5m 53.2M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 bf749144663e 2024-04-26T01:57:10.385 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi045 running (4m) 64s ago 4m 44.5M 4273M 19.0.0-3156-gd69f13d8 da4b004c09e8 d1b05e579912 2024-04-26T01:57:10.385 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi071 running (4m) 64s ago 4m 53.5M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b73a5bd49aa2 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.386+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7feba80a3c40 msgr2=0x7feba80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.386+0000 7febc8e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7feba80a3c40 0x7feba80a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7febc4102a00 tx=0x7febb4009260 comp rx=0 tx=0).stop 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.386+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 msgr2=0x7febc419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.386+0000 7febc8e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 0x7febc419b320 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7febb000e9f0 tx=0x7febb000ed00 comp rx=0 tx=0).stop 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 shutdown_connections 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febc4103540 0x7febc41a23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7feba80a3c40 0x7feba80a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7febc41019a0 0x7febc419ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 --2- 172.21.15.37:0/2989788470 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7febc4102ba0 0x7febc419b320 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 >> 172.21.15.37:0/2989788470 conn(0x7febc40fcf40 msgr2=0x7febc40fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 shutdown_connections 2024-04-26T01:57:10.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:10.387+0000 7febc8e00700 1 -- 172.21.15.37:0/2989788470 wait complete. 2024-04-26T01:57:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:09 smithi071 ceph-mon[32578]: pgmap v424: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:11.223 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T01:57:11.703 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:12.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:11 smithi037 ceph-mon[27906]: from='client.24863 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:12.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:11 smithi037 ceph-mon[27906]: pgmap v425: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:11 smithi045 ceph-mon[32520]: from='client.24863 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:11 smithi045 ceph-mon[32520]: pgmap v425: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:11 smithi071 ceph-mon[32578]: from='client.24863 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:11 smithi071 ceph-mon[32578]: pgmap v425: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 -- 172.21.15.37:0/454940913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104106880 msgr2=0x7f910410d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 --2- 172.21.15.37:0/454940913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104106880 0x7f910410d130 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00b0d0 tx=0x7f90fc00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 -- 172.21.15.37:0/454940913 shutdown_connections 2024-04-26T01:57:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 --2- 172.21.15.37:0/454940913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104106880 0x7f910410d130 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 --2- 172.21.15.37:0/454940913 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104104ce0 0x7f9104105100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 --2- 172.21.15.37:0/454940913 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9104105ee0 0x7f9104106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 -- 172.21.15.37:0/454940913 >> 172.21.15.37:0/454940913 conn(0x7f91041002a0 msgr2=0x7f91041026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.928+0000 7f910ae00700 1 -- 172.21.15.37:0/454940913 shutdown_connections 2024-04-26T01:57:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.929+0000 7f910ae00700 1 -- 172.21.15.37:0/454940913 wait complete. 2024-04-26T01:57:13.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.930+0000 7f910ae00700 1 Processor -- start 2024-04-26T01:57:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.930+0000 7f910ae00700 1 -- start start 2024-04-26T01:57:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.930+0000 7f910ae00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 0x7f910419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.930+0000 7f910ae00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104105ee0 0x7f910419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f910ae00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9104106880 0x7f91041a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f910ae00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9104110760 con 0x7f9104104ce0 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f910ae00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f91041105e0 con 0x7f9104106880 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f910ae00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f91041108e0 con 0x7f9104105ee0 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 0x7f910419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 0x7f910419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39260/0 (socket says 172.21.15.37:39260) 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104105ee0 0x7f910419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103e00700 1 -- 172.21.15.37:0/3464826684 learned_addr learned my addr 172.21.15.37:0/3464826684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9108e00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9104106880 0x7f91041a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103400700 1 -- 172.21.15.37:0/3464826684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9104106880 msgr2=0x7f91041a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103400700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9104106880 0x7f91041a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103400700 1 -- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 msgr2=0x7f910419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103400700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 0x7f910419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103400700 1 -- 172.21.15.37:0/3464826684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91041a2980 con 0x7f9104105ee0 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.931+0000 7f9103e00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 0x7f910419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:13.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.932+0000 7f9103400700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104105ee0 0x7f910419b270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9f0 tx=0x7f90f000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.932+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f000c9c0 con 0x7f9104105ee0 2024-04-26T01:57:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.932+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90f000cb60 con 0x7f9104105ee0 2024-04-26T01:57:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.932+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f0018680 con 0x7f9104105ee0 2024-04-26T01:57:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.932+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91041a2c70 con 0x7f9104105ee0 2024-04-26T01:57:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.933+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91041a31b0 con 0x7f9104105ee0 2024-04-26T01:57:13.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.934+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9104069730 con 0x7f9104105ee0 2024-04-26T01:57:13.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.936+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f90f0010430 con 0x7f9104105ee0 2024-04-26T01:57:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.937+0000 7f9100c00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90e80a3c40 0x7f90e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.937+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f90f00d05c0 con 0x7f9104105ee0 2024-04-26T01:57:13.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.937+0000 7f9103e00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90e80a3c40 0x7f90e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:13.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.938+0000 7f9103e00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90e80a3c40 0x7f90e80a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f910419c210 tx=0x7f90f400b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:13.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:13.942+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f90f009cf50 con 0x7f9104105ee0 2024-04-26T01:57:14.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.184+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f9104062e80 con 0x7f90e80a3c40 2024-04-26T01:57:14.183 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:13 smithi037 ceph-mon[27906]: pgmap v426: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:14.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.202+0000 7f9100c00700 1 -- 172.21.15.37:0/3464826684 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f9104062e80 con 0x7f90e80a3c40 2024-04-26T01:57:14.202 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T01:57:14.202 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 67s ago 4m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi045=b;smithi045=e;smithi045=h;smithi045=k;smithi071=c;smithi071=f;smithi071=i;smithi071=l;count:12 2024-04-26T01:57:14.202 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 67s ago 10m smithi037=x;smithi045=y;smithi071=z;count:3 2024-04-26T01:57:14.202 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 67s ago 10m smithi037:172.21.15.37=a;smithi045:172.21.15.45=b;smithi071:172.21.15.71=c;count:3 2024-04-26T01:57:14.202 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 67s ago - 2024-04-26T01:57:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90e80a3c40 msgr2=0x7f90e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90e80a3c40 0x7f90e80a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f910419c210 tx=0x7f90f400b4a0 comp rx=0 tx=0).stop 2024-04-26T01:57:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104105ee0 msgr2=0x7f910419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104105ee0 0x7f910419b270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f90f000e9f0 tx=0x7f90f000ed00 comp rx=0 tx=0).stop 2024-04-26T01:57:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 shutdown_connections 2024-04-26T01:57:14.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9104104ce0 0x7f910419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f90e80a3c40 0x7f90e80a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9104106880 0x7f91041a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 --2- 172.21.15.37:0/3464826684 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9104105ee0 0x7f910419b270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 >> 172.21.15.37:0/3464826684 conn(0x7f91041002a0 msgr2=0x7f91041091e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 shutdown_connections 2024-04-26T01:57:14.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:14.206+0000 7f910ae00700 1 -- 172.21.15.37:0/3464826684 wait complete. 2024-04-26T01:57:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:13 smithi045 ceph-mon[32520]: pgmap v426: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:13 smithi071 ceph-mon[32578]: pgmap v426: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:15.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T01:57:15.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:14 smithi037 ceph-mon[27906]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:15.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:14 smithi045 ceph-mon[32520]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:15.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:14 smithi071 ceph-mon[32578]: from='client.24910 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:15.473 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:15 smithi037 ceph-mon[27906]: pgmap v427: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:15 smithi045 ceph-mon[32520]: pgmap v427: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:15 smithi071 ceph-mon[32578]: pgmap v427: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 -- 172.21.15.37:0/2401682131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c103530 msgr2=0x7f734c105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 --2- 172.21.15.37:0/2401682131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c103530 0x7f734c105920 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f733c009990 tx=0x7f733c009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 -- 172.21.15.37:0/2401682131 shutdown_connections 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 --2- 172.21.15.37:0/2401682131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c103530 0x7f734c105920 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 --2- 172.21.15.37:0/2401682131 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734c108790 0x7f734c10ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 --2- 172.21.15.37:0/2401682131 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 -- 172.21.15.37:0/2401682131 >> 172.21.15.37:0/2401682131 conn(0x7f734c0fcf40 msgr2=0x7f734c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:17.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 -- 172.21.15.37:0/2401682131 shutdown_connections 2024-04-26T01:57:17.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.727+0000 7f7352e00700 1 -- 172.21.15.37:0/2401682131 wait complete. 2024-04-26T01:57:17.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 Processor -- start 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 -- start start 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734c103530 0x7f734c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c108790 0x7f734c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10e590 con 0x7f734c108790 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10e410 con 0x7f734c103530 2024-04-26T01:57:17.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.729+0000 7f7352e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f734c10e710 con 0x7f734c105e60 2024-04-26T01:57:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:17.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:58786/0 (socket says 172.21.15.37:58786) 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f734be00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734c103530 0x7f734c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 -- 172.21.15.37:0/2298299638 learned_addr learned my addr 172.21.15.37:0/2298299638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7350e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c108790 0x7f734c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 -- 172.21.15.37:0/2298299638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734c103530 msgr2=0x7f734c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734c103530 0x7f734c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 -- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c108790 msgr2=0x7f734c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c108790 0x7f734c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.730+0000 7f7343400700 1 -- 172.21.15.37:0/2298299638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c1a0930 con 0x7f734c105e60 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.731+0000 7f7343400700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c199220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f733400e9c0 tx=0x7f733400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.731+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733400cbd0 con 0x7f734c105e60 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.731+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c1a0c20 con 0x7f734c105e60 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.731+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733400ee60 con 0x7f734c105e60 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.731+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73340055c0 con 0x7f734c105e60 2024-04-26T01:57:17.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.731+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1a1160 con 0x7f734c105e60 2024-04-26T01:57:17.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.732+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f734c069730 con 0x7f734c105e60 2024-04-26T01:57:17.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.734+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7334004750 con 0x7f734c105e60 2024-04-26T01:57:17.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.735+0000 7f7349600700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f73300a3c40 0x7f73300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:17.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.736+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f73340d0700 con 0x7f734c105e60 2024-04-26T01:57:17.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.736+0000 7f734be00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f73300a3c40 0x7f73300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:17.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.737+0000 7f734be00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f73300a3c40 0x7f73300a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f734c19a1c0 tx=0x7f733c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:17.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.740+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f733409d090 con 0x7f734c105e60 2024-04-26T01:57:17.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.979+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f734c062e80 con 0x7f73300a3c40 2024-04-26T01:57:17.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.981+0000 7f7349600700 1 -- 172.21.15.37:0/2298299638 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+187 (secure 0 0 0) 0x7f734c062e80 con 0x7f73300a3c40 2024-04-26T01:57:17.981 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-04-26T01:57:17.981 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-04-26T01:57:17.981 INFO:teuthology.orchestra.run.smithi037.stdout:smithi045 172.21.15.45 2024-04-26T01:57:17.981 INFO:teuthology.orchestra.run.smithi037.stdout:smithi071 172.21.15.71 2024-04-26T01:57:17.981 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-04-26T01:57:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f73300a3c40 msgr2=0x7f73300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f73300a3c40 0x7f73300a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f734c19a1c0 tx=0x7f733c01a040 comp rx=0 tx=0).stop 2024-04-26T01:57:17.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 msgr2=0x7f734c199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c199220 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f733400e9c0 tx=0x7f733400ecd0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 shutdown_connections 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f734c108790 0x7f734c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f73300a3c40 0x7f73300a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f734c103530 0x7f734c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 --2- 172.21.15.37:0/2298299638 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f734c105e60 0x7f734c199220 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.984+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 >> 172.21.15.37:0/2298299638 conn(0x7f734c0fcf40 msgr2=0x7f734c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:17.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.985+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 shutdown_connections 2024-04-26T01:57:17.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:17.985+0000 7f7352e00700 1 -- 172.21.15.37:0/2298299638 wait complete. 2024-04-26T01:57:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:17 smithi037 ceph-mon[27906]: pgmap v428: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:17 smithi045 ceph-mon[32520]: pgmap v428: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:17 smithi071 ceph-mon[32578]: pgmap v428: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:18.875 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T01:57:19.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:18 smithi037 ceph-mon[27906]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:19.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:18 smithi045 ceph-mon[32520]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:19.317 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:19.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:18 smithi071 ceph-mon[32578]: from='client.24916 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:20.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:19 smithi037 ceph-mon[27906]: pgmap v429: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:19 smithi045 ceph-mon[32520]: pgmap v429: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:19 smithi071 ceph-mon[32578]: pgmap v429: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.592+0000 7f2bbd000700 1 -- 172.21.15.37:0/1767567852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8103bc0 msgr2=0x7f2bb8106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.592+0000 7f2bbd000700 1 --2- 172.21.15.37:0/1767567852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8103bc0 0x7f2bb8106050 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009990 tx=0x7f2ba4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 -- 172.21.15.37:0/1767567852 shutdown_connections 2024-04-26T01:57:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 --2- 172.21.15.37:0/1767567852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb8106a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 --2- 172.21.15.37:0/1767567852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb8101250 0x7f2bb8103680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 --2- 172.21.15.37:0/1767567852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8103bc0 0x7f2bb8106050 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 -- 172.21.15.37:0/1767567852 >> 172.21.15.37:0/1767567852 conn(0x7f2bb80fce70 msgr2=0x7f2bb80ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 -- 172.21.15.37:0/1767567852 shutdown_connections 2024-04-26T01:57:21.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.593+0000 7f2bbd000700 1 -- 172.21.15.37:0/1767567852 wait complete. 2024-04-26T01:57:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 Processor -- start 2024-04-26T01:57:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 -- start start 2024-04-26T01:57:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8101250 0x7f2bb819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb8103bc0 0x7f2bb819f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb81a6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:21.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb81149c0 con 0x7f2bb8106590 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8114840 con 0x7f2bb8101250 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.594+0000 7f2bbd000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb8114b40 con 0x7f2bb8103bc0 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb81a6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb81a6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41528/0 (socket says 172.21.15.37:41528) 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 -- 172.21.15.37:0/3722809578 learned_addr learned my addr 172.21.15.37:0/3722809578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb6a00700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb8103bc0 0x7f2bb819f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7400700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8101250 0x7f2bb819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 -- 172.21.15.37:0/3722809578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb8103bc0 msgr2=0x7f2bb819f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb8103bc0 0x7f2bb819f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 -- 172.21.15.37:0/3722809578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8101250 msgr2=0x7f2bb819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8101250 0x7f2bb819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.595+0000 7f2bb7e00700 1 -- 172.21.15.37:0/3722809578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb81a6cd0 con 0x7f2bb8106590 2024-04-26T01:57:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.596+0000 7f2bb7e00700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb81a6630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000ef60 tx=0x7f2bb000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.596+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb00092e0 con 0x7f2bb8106590 2024-04-26T01:57:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.596+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bb0004bb0 con 0x7f2bb8106590 2024-04-26T01:57:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.596+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb81a6f60 con 0x7f2bb8106590 2024-04-26T01:57:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.596+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb0004280 con 0x7f2bb8106590 2024-04-26T01:57:21.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.596+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb81a73c0 con 0x7f2bb8106590 2024-04-26T01:57:21.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.597+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bb8069730 con 0x7f2bb8106590 2024-04-26T01:57:21.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.599+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2bb001b030 con 0x7f2bb8106590 2024-04-26T01:57:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.602+0000 7f2bafe00700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b9c0a3c40 0x7f2b9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.602+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f2bb00cff10 con 0x7f2bb8106590 2024-04-26T01:57:21.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.602+0000 7f2bb7400700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b9c0a3c40 0x7f2b9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:21.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.603+0000 7f2bb7400700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b9c0a3c40 0x7f2b9c0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009bf0 tx=0x7f2ba8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:21.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.607+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bb0022020 con 0x7f2bb8106590 2024-04-26T01:57:21.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.855+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f2bb8062e80 con 0x7f2b9c0a3c40 2024-04-26T01:57:21.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.861+0000 7f2bafe00700 1 -- 172.21.15.37:0/3722809578 <== mgr.14152 v2:172.21.15.37:6800/691602839 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f2bb8062e80 con 0x7f2b9c0a3c40 2024-04-26T01:57:21.860 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T01:57:21.860 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-04-26T01:57:21.860 INFO:teuthology.orchestra.run.smithi037.stdout:smithi045 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120098480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-04-26T01:57:21.860 INFO:teuthology.orchestra.run.smithi037.stdout:smithi071 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040172400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T01:57:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b9c0a3c40 msgr2=0x7f2b9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b9c0a3c40 0x7f2b9c0a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2ba8009bf0 tx=0x7f2ba8009260 comp rx=0 tx=0).stop 2024-04-26T01:57:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 msgr2=0x7f2bb81a6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:21.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb81a6630 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2bb000ef60 tx=0x7f2bb000ef90 comp rx=0 tx=0).stop 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 shutdown_connections 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bb8106590 0x7f2bb81a6630 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f2b9c0a3c40 0x7f2b9c0a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2bb8101250 0x7f2bb819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 --2- 172.21.15.37:0/3722809578 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2bb8103bc0 0x7f2bb819f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 >> 172.21.15.37:0/3722809578 conn(0x7f2bb80fce70 msgr2=0x7f2bb8101fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 shutdown_connections 2024-04-26T01:57:21.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:21.864+0000 7f2bbd000700 1 -- 172.21.15.37:0/3722809578 wait complete. 2024-04-26T01:57:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:21 smithi037 ceph-mon[27906]: pgmap v430: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:21 smithi045 ceph-mon[32520]: pgmap v430: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:21 smithi071 ceph-mon[32578]: pgmap v430: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:22.716 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T01:57:22.731 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T01:57:22.731 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T01:57:23.214 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:23.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:22 smithi045 ceph-mon[32520]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:23.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:22 smithi037 ceph-mon[27906]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:23.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:22 smithi071 ceph-mon[32578]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:24.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:23 smithi045 ceph-mon[32520]: pgmap v431: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:24.303 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:23 smithi037 ceph-mon[27906]: pgmap v431: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:23 smithi071 ceph-mon[32578]: pgmap v431: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 -- 172.21.15.37:0/1986883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 msgr2=0x7f47c8075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 --2- 172.21.15.37:0/1986883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c8075920 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098f0 tx=0x7f47b0009c00 comp rx=0 tx=0).stop 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 -- 172.21.15.37:0/1986883601 shutdown_connections 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 --2- 172.21.15.37:0/1986883601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c810a980 0x7f47c810ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 --2- 172.21.15.37:0/1986883601 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c8076a70 0x7f47c8074ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 --2- 172.21.15.37:0/1986883601 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c8075920 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 -- 172.21.15.37:0/1986883601 >> 172.21.15.37:0/1986883601 conn(0x7f47c80fe010 msgr2=0x7f47c8100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 -- 172.21.15.37:0/1986883601 shutdown_connections 2024-04-26T01:57:25.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.524+0000 7f47ce400700 1 -- 172.21.15.37:0/1986883601 wait complete. 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.525+0000 7f47ce400700 1 Processor -- start 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.525+0000 7f47ce400700 1 -- start start 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47ce400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47ce400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 0x7f47c819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47ce400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c810a980 0x7f47c81a2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47ce400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81107b0 con 0x7f47c8076a70 2024-04-26T01:57:25.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47ce400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8110630 con 0x7f47c810a980 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47ce400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8110930 con 0x7f47c80754a0 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 0x7f47c819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44732/0 (socket says 172.21.15.37:44732) 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7400700 1 -- 172.21.15.37:0/1973877497 learned_addr learned my addr 172.21.15.37:0/1973877497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47bea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 0x7f47c819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41546/0 (socket says 172.21.15.37:41546) 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7e00700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c810a980 0x7f47c81a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7400700 1 -- 172.21.15.37:0/1973877497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c810a980 msgr2=0x7f47c81a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c810a980 0x7f47c81a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.526+0000 7f47c7400700 1 -- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 msgr2=0x7f47c819b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47c7400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 0x7f47c819b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47c7400700 1 -- 172.21.15.37:0/1973877497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81a2af0 con 0x7f47c80754a0 2024-04-26T01:57:25.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47bea00700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 0x7f47c819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47c7400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c819ae90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47b800efc0 tx=0x7f47b800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b80092e0 con 0x7f47c80754a0 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47b8004bb0 con 0x7f47c80754a0 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c81a2d80 con 0x7f47c80754a0 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8004280 con 0x7f47c80754a0 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.527+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a31b0 con 0x7f47c80754a0 2024-04-26T01:57:25.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.528+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8069730 con 0x7f47c80754a0 2024-04-26T01:57:25.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.530+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f47b801b030 con 0x7f47c80754a0 2024-04-26T01:57:25.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.532+0000 7f47c4c00700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47ac0a3c40 0x7f47ac0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:25.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.532+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f47b80cff80 con 0x7f47c80754a0 2024-04-26T01:57:25.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.532+0000 7f47bea00700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47ac0a3c40 0x7f47ac0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:25.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.533+0000 7f47bea00700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47ac0a3c40 0x7f47ac0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098c0 tx=0x7f47b0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:25.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.536+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47b8022020 con 0x7f47c80754a0 2024-04-26T01:57:25.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.819+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f47c80636f0 con 0x7f47c80754a0 2024-04-26T01:57:25.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.821+0000 7f47c4c00700 1 -- 172.21.15.37:0/1973877497 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 19 v19) ==== 76+0+2991 (secure 0 0 0) 0x7f47b8010040 con 0x7f47c80754a0 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:e19 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 19 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T01:57:25.821 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-26T01:54:12.162501+0000 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-26T01:54:33.402956+0000 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-04-26T01:57:25.822 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 1 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:in 0 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24514} 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-04-26T01:57:25.823 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24514 members: 24514 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{0:24514} state up:active seq 11 addr [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{0:24509} state up:standby-replay seq 1 addr [v2:172.21.15.45:6838/60713297,v1:172.21.15.45:6839/60713297] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{-1:14403} state up:standby seq 1 addr [v2:172.21.15.37:6840/2822754454,v1:172.21.15.37:6841/2822754454] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{-1:24428} state up:standby seq 1 addr [v2:172.21.15.45:6832/2132222354,v1:172.21.15.45:6833/2132222354] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:24440} state up:standby seq 1 addr [v2:172.21.15.37:6834/3616070379,v1:172.21.15.37:6835/3616070379] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:24446} state up:standby seq 1 addr [v2:172.21.15.37:6836/200842518,v1:172.21.15.37:6837/200842518] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.71:6832/10421136,v1:172.21.15.71:6833/10421136] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{-1:24458} state up:standby seq 1 addr [v2:172.21.15.45:6834/3667220956,v1:172.21.15.45:6835/3667220956] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{-1:24470} state up:standby seq 1 addr [v2:172.21.15.45:6836/1744923688,v1:172.21.15.45:6837/1744923688] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:24485} state up:standby seq 1 addr [v2:172.21.15.37:6838/1856429738,v1:172.21.15.37:6839/1856429738] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.824 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{-1:24490} state up:standby seq 1 addr [v2:172.21.15.71:6834/1784845440,v1:172.21.15.71:6835/1784845440] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.825 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{-1:24502} state up:standby seq 1 addr [v2:172.21.15.71:6836/306008379,v1:172.21.15.71:6837/306008379] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T01:57:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.823+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47ac0a3c40 msgr2=0x7f47ac0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.823+0000 7f47ce400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47ac0a3c40 0x7f47ac0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f47b00098c0 tx=0x7f47b0019040 comp rx=0 tx=0).stop 2024-04-26T01:57:25.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.823+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 msgr2=0x7f47c819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.823+0000 7f47ce400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c819ae90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47b800efc0 tx=0x7f47b800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 shutdown_connections 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f47ac0a3c40 0x7f47ac0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47c810a980 0x7f47c81a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47c80754a0 0x7f47c819ae90 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f47b800efc0 tx=0x7f47b800c3a0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 --2- 172.21.15.37:0/1973877497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8076a70 0x7f47c819b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 >> 172.21.15.37:0/1973877497 conn(0x7f47c80fe010 msgr2=0x7f47c8106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 shutdown_connections 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:25.824+0000 7f47ce400700 1 -- 172.21.15.37:0/1973877497 wait complete. 2024-04-26T01:57:25.826 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:57:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:25 smithi037 ceph-mon[27906]: pgmap v432: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:25 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1973877497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:57:26.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:25 smithi045 ceph-mon[32520]: pgmap v432: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:26.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:25 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1973877497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:57:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:25 smithi071 ceph-mon[32578]: pgmap v432: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:25 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1973877497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T01:57:26.807 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T01:57:27.277 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:27 smithi037 ceph-mon[27906]: pgmap v433: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:27 smithi045 ceph-mon[32520]: pgmap v433: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:27 smithi071 ceph-mon[32578]: pgmap v433: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 -- 172.21.15.37:0/895461204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec076a10 msgr2=0x7f62ec074e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 --2- 172.21.15.37:0/895461204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec076a10 0x7f62ec074e70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b0a0 tx=0x7f62dc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 -- 172.21.15.37:0/895461204 shutdown_connections 2024-04-26T01:57:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 --2- 172.21.15.37:0/895461204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec076a10 0x7f62ec074e70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 --2- 172.21.15.37:0/895461204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec10a980 0x7f62ec10ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 --2- 172.21.15.37:0/895461204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec075440 0x7f62ec0758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.542+0000 7f62f3c00700 1 -- 172.21.15.37:0/895461204 >> 172.21.15.37:0/895461204 conn(0x7f62ec0fe040 msgr2=0x7f62ec100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:29.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.543+0000 7f62f3c00700 1 -- 172.21.15.37:0/895461204 shutdown_connections 2024-04-26T01:57:29.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.543+0000 7f62f3c00700 1 -- 172.21.15.37:0/895461204 wait complete. 2024-04-26T01:57:29.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 Processor -- start 2024-04-26T01:57:29.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 -- start start 2024-04-26T01:57:29.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 0x7f62ec19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:29.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec076a10 0x7f62ec19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec10a980 0x7f62ec1a2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec110710 con 0x7f62ec075440 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec110590 con 0x7f62ec076a10 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f3c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec110890 con 0x7f62ec10a980 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.544+0000 7f62f1200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 0x7f62ec19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 0x7f62ec19add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41554/0 (socket says 172.21.15.37:41554) 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 -- 172.21.15.37:0/60001659 learned_addr learned my addr 172.21.15.37:0/60001659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62e3e00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec076a10 0x7f62ec19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec10a980 0x7f62ec1a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 -- 172.21.15.37:0/60001659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec10a980 msgr2=0x7f62ec1a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec10a980 0x7f62ec1a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 -- 172.21.15.37:0/60001659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec076a10 msgr2=0x7f62ec19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:29.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec076a10 0x7f62ec19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 -- 172.21.15.37:0/60001659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec1a2a30 con 0x7f62ec075440 2024-04-26T01:57:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.545+0000 7f62f1200700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 0x7f62ec19add0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b070 tx=0x7f62dc004d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:29.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.546+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc021070 con 0x7f62ec075440 2024-04-26T01:57:29.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.546+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62dc007b80 con 0x7f62ec075440 2024-04-26T01:57:29.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.546+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62dc0052a0 con 0x7f62ec075440 2024-04-26T01:57:29.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.546+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec1a2cc0 con 0x7f62ec075440 2024-04-26T01:57:29.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.546+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec1a3180 con 0x7f62ec075440 2024-04-26T01:57:29.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.547+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec069730 con 0x7f62ec075440 2024-04-26T01:57:29.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.549+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f62dc024070 con 0x7f62ec075440 2024-04-26T01:57:29.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.550+0000 7f62ea000700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f62d00a3cc0 0x7f62d00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:29.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.551+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f62dc00f020 con 0x7f62ec075440 2024-04-26T01:57:29.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.551+0000 7f62e3e00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f62d00a3cc0 0x7f62d00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:29.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.551+0000 7f62e3e00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f62d00a3cc0 0x7f62d00a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009c60 tx=0x7f62d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:29.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.555+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f62dc02b020 con 0x7f62ec075440 2024-04-26T01:57:29.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.788+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f62ec063650 con 0x7f62ec075440 2024-04-26T01:57:29.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.791+0000 7f62ea000700 1 -- 172.21.15.37:0/60001659 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v91) ==== 56+0+5250 (secure 0 0 0) 0x7f62dc09e360 con 0x7f62ec075440 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 91 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:fsid 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-04-26T01:44:24.368582+0000 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-04-26T01:54:44.157091+0000 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-04-26T01:57:29.791 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/2603696107,v1:172.21.15.37:6803/2603696107] [v2:172.21.15.37:6804/2603696107,v1:172.21.15.37:6805/2603696107] exists,up d2501dac-396b-46d1-a9ff-b3b157f46fb4 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6800/3445601743,v1:172.21.15.45:6801/3445601743] [v2:172.21.15.45:6802/3445601743,v1:172.21.15.45:6803/3445601743] exists,up f91f576c-313e-4a6a-9f40-715003e44614 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6800/2117420904,v1:172.21.15.71:6801/2117420904] [v2:172.21.15.71:6802/2117420904,v1:172.21.15.71:6803/2117420904] exists,up a90c81da-6303-4de4-8556-bd4ba8819380 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 26 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/1364499225,v1:172.21.15.37:6811/1364499225] [v2:172.21.15.37:6812/1364499225,v1:172.21.15.37:6813/1364499225] exists,up 950c30ee-de7f-461e-89b8-5218e34caf03 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6808/2655399614,v1:172.21.15.45:6809/2655399614] [v2:172.21.15.45:6810/2655399614,v1:172.21.15.45:6811/2655399614] exists,up b70dc87f-17e7-44e4-b5d1-80abdcc91aed 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 39 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6808/1060444370,v1:172.21.15.71:6809/1060444370] [v2:172.21.15.71:6810/1060444370,v1:172.21.15.71:6811/1060444370] exists,up bbe95809-3e82-4167-8b84-fed9410871e9 2024-04-26T01:57:29.792 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/2087418626,v1:172.21.15.37:6819/2087418626] [v2:172.21.15.37:6820/2087418626,v1:172.21.15.37:6821/2087418626] exists,up 1af9e89d-771d-4ce4-8e7b-693cf99fe580 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 52 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6816/1922916284,v1:172.21.15.45:6817/1922916284] [v2:172.21.15.45:6818/1922916284,v1:172.21.15.45:6819/1922916284] exists,up 3389b007-2053-4508-a620-d820a3a6fecf 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 60 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6816/1870194976,v1:172.21.15.71:6817/1870194976] [v2:172.21.15.71:6818/1870194976,v1:172.21.15.71:6819/1870194976] exists,up ad43bcc8-3414-401c-bbff-6ff4ed6a3149 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 68 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/759844369,v1:172.21.15.37:6827/759844369] [v2:172.21.15.37:6828/759844369,v1:172.21.15.37:6829/759844369] exists,up 22321723-8f84-485f-9ecc-fe50bf7c0233 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 73 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.45:6824/685235044,v1:172.21.15.45:6825/685235044] [v2:172.21.15.45:6826/685235044,v1:172.21.15.45:6827/685235044] exists,up f2b2d29e-56a1-427e-a87f-911da6c8adf4 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 81 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.71:6824/584658428,v1:172.21.15.71:6825/584658428] [v2:172.21.15.71:6826/584658428,v1:172.21.15.71:6827/584658428] exists,up 16fcf494-d10b-4505-a211-636f44fcf2f1 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.6 [8,11] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.8 [2,11] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.10 [2,4] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.6 [4,7] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.b [3,9] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.20 [5,9] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.27 [4,8] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.2c [4,9] 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2267935573 expires 2024-04-27T01:45:42.363226+0000 2024-04-26T01:57:29.793 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1381224435 expires 2024-04-27T01:45:42.363226+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2267935573 expires 2024-04-27T01:45:42.363226+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3897485386 expires 2024-04-27T01:45:42.363226+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/1210493699 expires 2024-04-27T01:45:06.740013+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/1210493699 expires 2024-04-27T01:45:06.740013+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2037138951 expires 2024-04-27T01:45:06.740013+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/19474513 expires 2024-04-27T01:45:06.740013+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3832279297 expires 2024-04-27T01:45:42.363226+0000 2024-04-26T01:57:29.794 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2777822460 expires 2024-04-27T01:45:06.740013+0000 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.795+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f62d00a3cc0 msgr2=0x7f62d00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.795+0000 7f62f3c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f62d00a3cc0 0x7f62d00a6180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f62d4009c60 tx=0x7f62d4009260 comp rx=0 tx=0).stop 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.795+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 msgr2=0x7f62ec19add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.795+0000 7f62f3c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 0x7f62ec19add0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f62dc00b070 tx=0x7f62dc004d70 comp rx=0 tx=0).stop 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.795+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 shutdown_connections 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.795+0000 7f62f3c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62ec075440 0x7f62ec19add0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.796+0000 7f62f3c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f62d00a3cc0 0x7f62d00a6180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.796+0000 7f62f3c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f62ec10a980 0x7f62ec1a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.796+0000 7f62f3c00700 1 --2- 172.21.15.37:0/60001659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f62ec076a10 0x7f62ec19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:29.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.796+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 >> 172.21.15.37:0/60001659 conn(0x7f62ec0fe040 msgr2=0x7f62ec1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.796+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 shutdown_connections 2024-04-26T01:57:29.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:29.796+0000 7f62f3c00700 1 -- 172.21.15.37:0/60001659 wait complete. 2024-04-26T01:57:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:29 smithi037 ceph-mon[27906]: pgmap v434: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:29 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/60001659' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:57:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:29 smithi045 ceph-mon[32520]: pgmap v434: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:29 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/60001659' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:57:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:29 smithi071 ceph-mon[32578]: pgmap v434: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:29 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/60001659' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T01:57:30.674 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T01:57:30.691 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:57:31.154 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:31 smithi045 ceph-mon[32520]: pgmap v435: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:32.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:31 smithi037 ceph-mon[27906]: pgmap v435: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:31 smithi071 ceph-mon[32578]: pgmap v435: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 -- 172.21.15.37:0/1580373075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c06a750 msgr2=0x7f541c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 --2- 172.21.15.37:0/1580373075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c06a750 0x7f541c06abb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5408009990 tx=0x7f5408009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 -- 172.21.15.37:0/1580373075 shutdown_connections 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 --2- 172.21.15.37:0/1580373075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06b0f0 0x7f541c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 --2- 172.21.15.37:0/1580373075 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f541c108a30 0x7f541c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 --2- 172.21.15.37:0/1580373075 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c06a750 0x7f541c06abb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.431+0000 7f5423c00700 1 -- 172.21.15.37:0/1580373075 >> 172.21.15.37:0/1580373075 conn(0x7f541c0788f0 msgr2=0x7f541c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.432+0000 7f5423c00700 1 -- 172.21.15.37:0/1580373075 shutdown_connections 2024-04-26T01:57:33.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.432+0000 7f5423c00700 1 -- 172.21.15.37:0/1580373075 wait complete. 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.433+0000 7f5423c00700 1 Processor -- start 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.433+0000 7f5423c00700 1 -- start start 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5423c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06a750 0x7f541c199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5423c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f541c06b0f0 0x7f541c199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5423c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c108a30 0x7f541c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5423c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f541c112c40 con 0x7f541c06a750 2024-04-26T01:57:33.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5423c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f541c112ac0 con 0x7f541c06b0f0 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5423c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f541c112dc0 con 0x7f541c108a30 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5421200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06a750 0x7f541c199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5421200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06a750 0x7f541c199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58864/0 (socket says 172.21.15.37:58864) 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5421200700 1 -- 172.21.15.37:0/503570227 learned_addr learned my addr 172.21.15.37:0/503570227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f5421c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c108a30 0x7f541c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.434+0000 7f541be00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f541c06b0f0 0x7f541c199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5421c00700 1 -- 172.21.15.37:0/503570227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f541c06b0f0 msgr2=0x7f541c199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5421c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f541c06b0f0 0x7f541c199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5421c00700 1 -- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06a750 msgr2=0x7f541c199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5421c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06a750 0x7f541c199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5421c00700 1 -- 172.21.15.37:0/503570227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f541c19e160 con 0x7f541c108a30 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5421c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c108a30 0x7f541c19da20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f541400efc0 tx=0x7f541400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.435+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54140092e0 con 0x7f541c108a30 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.436+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5414004bb0 con 0x7f541c108a30 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.436+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5414004020 con 0x7f541c108a30 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.436+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f541c19e3f0 con 0x7f541c108a30 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.436+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f541c1a5cd0 con 0x7f541c108a30 2024-04-26T01:57:33.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.437+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f541c069730 con 0x7f541c108a30 2024-04-26T01:57:33.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.440+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f541401b030 con 0x7f541c108a30 2024-04-26T01:57:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.441+0000 7f5419600700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f54040a3cc0 0x7f54040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.441+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f54140d0070 con 0x7f541c108a30 2024-04-26T01:57:33.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.441+0000 7f5421200700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f54040a3cc0 0x7f54040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:33.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.442+0000 7f5421200700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f54040a3cc0 0x7f54040a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5410009ca0 tx=0x7f5410009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:33.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.443+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5414022020 con 0x7f541c108a30 2024-04-26T01:57:33.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.728+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f541c19a4b0 con 0x7f541c108a30 2024-04-26T01:57:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:33.730 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:57:33.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.730+0000 7f5419600700 1 -- 172.21.15.37:0/503570227 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f5414010040 con 0x7f541c108a30 2024-04-26T01:57:33.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f54040a3cc0 msgr2=0x7f54040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:33.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f54040a3cc0 0x7f54040a6180 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5410009ca0 tx=0x7f5410009260 comp rx=0 tx=0).stop 2024-04-26T01:57:33.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c108a30 msgr2=0x7f541c19da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:33.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c108a30 0x7f541c19da20 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f541400efc0 tx=0x7f541400c3a0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 shutdown_connections 2024-04-26T01:57:33.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f541c06a750 0x7f541c199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f54040a3cc0 0x7f54040a6180 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f541c108a30 0x7f541c19da20 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 --2- 172.21.15.37:0/503570227 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f541c06b0f0 0x7f541c199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:33.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 >> 172.21.15.37:0/503570227 conn(0x7f541c0788f0 msgr2=0x7f541c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:33.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 shutdown_connections 2024-04-26T01:57:33.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:33.733+0000 7f5423c00700 1 -- 172.21.15.37:0/503570227 wait complete. 2024-04-26T01:57:33.736 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:57:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:34 smithi045 ceph-mon[32520]: pgmap v436: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:34 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/503570227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:34 smithi071 ceph-mon[32578]: pgmap v436: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:34 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/503570227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:34.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:34 smithi037 ceph-mon[27906]: pgmap v436: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:34.416 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:34 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/503570227' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:34.473 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:57:34.938 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:36.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:36 smithi037 ceph-mon[27906]: pgmap v437: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:36.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:36.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.386 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:36 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:36 smithi071 ceph-mon[32578]: pgmap v437: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:36 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:36 smithi045 ceph-mon[32520]: pgmap v437: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:36 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:37.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:37 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:37.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:37 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:37.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:37 smithi037 ceph-mon[27906]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 -- 172.21.15.37:0/1733308708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 msgr2=0x7fe49c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 --2- 172.21.15.37:0/1733308708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c101dc0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe488009990 tx=0x7fe488009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 -- 172.21.15.37:0/1733308708 shutdown_connections 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 --2- 172.21.15.37:0/1733308708 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 0x7fe49c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 --2- 172.21.15.37:0/1733308708 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49c107990 0x7fe49c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 --2- 172.21.15.37:0/1733308708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c101dc0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 -- 172.21.15.37:0/1733308708 >> 172.21.15.37:0/1733308708 conn(0x7fe49c0788f0 msgr2=0x7fe49c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 -- 172.21.15.37:0/1733308708 shutdown_connections 2024-04-26T01:57:37.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.263+0000 7fe4a4200700 1 -- 172.21.15.37:0/1733308708 wait complete. 2024-04-26T01:57:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.264+0000 7fe4a4200700 1 Processor -- start 2024-04-26T01:57:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.264+0000 7fe4a4200700 1 -- start start 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a4200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a4200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 0x7fe49c199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a4200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49c107990 0x7fe49c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a4200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c111b50 con 0x7fe49c101940 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a4200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c1119d0 con 0x7fe49c102300 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a4200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe49c111cd0 con 0x7fe49c107990 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 0x7fe49c199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a2200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49c107990 0x7fe49c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a1800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58888/0 (socket says 172.21.15.37:58888) 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a1800700 1 -- 172.21.15.37:0/415156397 learned_addr learned my addr 172.21.15.37:0/415156397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.265+0000 7fe4a0e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 0x7fe49c199730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:37020/0 (socket says 172.21.15.37:37020) 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe4a1800700 1 -- 172.21.15.37:0/415156397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49c107990 msgr2=0x7fe49c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe4a1800700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49c107990 0x7fe49c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe4a1800700 1 -- 172.21.15.37:0/415156397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 msgr2=0x7fe49c199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe4a1800700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 0x7fe49c199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe4a1800700 1 -- 172.21.15.37:0/415156397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49c19e1b0 con 0x7fe49c101940 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe4a1800700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c1991f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe49000e9c0 tx=0x7fe49000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.266+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49000cbd0 con 0x7fe49c101940 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.267+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe49000ee50 con 0x7fe49c101940 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.267+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490005550 con 0x7fe49c101940 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.267+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49c19e4a0 con 0x7fe49c101940 2024-04-26T01:57:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.267+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe49c1a5d20 con 0x7fe49c101940 2024-04-26T01:57:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.268+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe49c069730 con 0x7fe49c101940 2024-04-26T01:57:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.270+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe490004750 con 0x7fe49c101940 2024-04-26T01:57:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.271+0000 7fe49a000700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe4840a3cc0 0x7fe4840a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.271+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fe4900d05a0 con 0x7fe49c101940 2024-04-26T01:57:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.272+0000 7fe4a0e00700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe4840a3cc0 0x7fe4840a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.272+0000 7fe4a0e00700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe4840a3cc0 0x7fe4840a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe49c079110 tx=0x7fe48800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.276+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe49009cf30 con 0x7fe49c101940 2024-04-26T01:57:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:37 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:37 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:37.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:37 smithi071 ceph-mon[32578]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:37 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:57:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:37 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:57:37.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:37 smithi045 ceph-mon[32520]: from='mgr.14152 172.21.15.37:0/1316614473' entity='mgr.x' 2024-04-26T01:57:37.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.554+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe49c02df80 con 0x7fe49c101940 2024-04-26T01:57:37.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.556+0000 7fe49a000700 1 -- 172.21.15.37:0/415156397 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fe49009c830 con 0x7fe49c101940 2024-04-26T01:57:37.555 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:37.555 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:57:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe4840a3cc0 msgr2=0x7fe4840a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe4840a3cc0 0x7fe4840a6180 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe49c079110 tx=0x7fe48800b260 comp rx=0 tx=0).stop 2024-04-26T01:57:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 msgr2=0x7fe49c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c1991f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe49000e9c0 tx=0x7fe49000ecd0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 shutdown_connections 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe49c101940 0x7fe49c1991f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fe4840a3cc0 0x7fe4840a6180 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe49c107990 0x7fe49c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 --2- 172.21.15.37:0/415156397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe49c102300 0x7fe49c199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 >> 172.21.15.37:0/415156397 conn(0x7fe49c0788f0 msgr2=0x7fe49c06aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 shutdown_connections 2024-04-26T01:57:37.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:37.560+0000 7fe4a4200700 1 -- 172.21.15.37:0/415156397 wait complete. 2024-04-26T01:57:37.561 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:57:38.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:38 smithi037 ceph-mon[27906]: pgmap v438: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:38.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:38 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/415156397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:38.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:57:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:38 smithi071 ceph-mon[32578]: pgmap v438: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:38 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/415156397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:38 smithi045 ceph-mon[32520]: pgmap v438: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:38 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/415156397' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:57:38.796 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:40.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:40 smithi037 ceph-mon[27906]: pgmap v439: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:40.306 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:40 smithi045 ceph-mon[32520]: pgmap v439: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:40 smithi071 ceph-mon[32578]: pgmap v439: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:41.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 -- 172.21.15.37:0/2405407710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708108a50 msgr2=0x7fb708108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:41.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 --2- 172.21.15.37:0/2405407710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708108a50 0x7fb708108e30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb6f800b0a0 tx=0x7fb6f800b3b0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 -- 172.21.15.37:0/2405407710 shutdown_connections 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 --2- 172.21.15.37:0/2405407710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708108a50 0x7fb708108e30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 --2- 172.21.15.37:0/2405407710 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708103450 0x7fb70810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 --2- 172.21.15.37:0/2405407710 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708102a90 0x7fb708102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 -- 172.21.15.37:0/2405407710 >> 172.21.15.37:0/2405407710 conn(0x7fb708079ba0 msgr2=0x7fb708079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 -- 172.21.15.37:0/2405407710 shutdown_connections 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.042+0000 7fb70e200700 1 -- 172.21.15.37:0/2405407710 wait complete. 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.043+0000 7fb70e200700 1 Processor -- start 2024-04-26T01:57:41.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.043+0000 7fb70e200700 1 -- start start 2024-04-26T01:57:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb70e200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708102a90 0x7fb70819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb70e200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 0x7fb70819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb70e200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708108a50 0x7fb70819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb70e200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112be0 con 0x7fb708102a90 2024-04-26T01:57:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb70e200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112a60 con 0x7fb708108a50 2024-04-26T01:57:41.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb70e200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb708112d60 con 0x7fb708103450 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 0x7fb70819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb707400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708102a90 0x7fb70819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb6fea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 0x7fb70819b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44200/0 (socket says 172.21.15.37:44200) 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb6fea00700 1 -- 172.21.15.37:0/2510223338 learned_addr learned my addr 172.21.15.37:0/2510223338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.044+0000 7fb707e00700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708108a50 0x7fb70819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb6fea00700 1 -- 172.21.15.37:0/2510223338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708108a50 msgr2=0x7fb70819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:41.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb6fea00700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708108a50 0x7fb70819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb6fea00700 1 -- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708102a90 msgr2=0x7fb70819b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb6fea00700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708102a90 0x7fb70819b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb6fea00700 1 -- 172.21.15.37:0/2510223338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7081a0340 con 0x7fb708103450 2024-04-26T01:57:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb707400700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708102a90 0x7fb70819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb6fea00700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 0x7fb70819b870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000d980 tx=0x7fb6f000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:41.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0016070 con 0x7fb708103450 2024-04-26T01:57:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6f0004830 con 0x7fb708103450 2024-04-26T01:57:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.045+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7081a0630 con 0x7fb708103450 2024-04-26T01:57:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.046+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6f0005540 con 0x7fb708103450 2024-04-26T01:57:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.046+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7081a7eb0 con 0x7fb708103450 2024-04-26T01:57:41.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.047+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb708069730 con 0x7fb708103450 2024-04-26T01:57:41.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.049+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb6f0004220 con 0x7fb708103450 2024-04-26T01:57:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.050+0000 7fb704c00700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb6ec0a3cc0 0x7fb6ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:41.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.050+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7fb6f00d01a0 con 0x7fb708103450 2024-04-26T01:57:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.050+0000 7fb707400700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb6ec0a3cc0 0x7fb6ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:41.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.051+0000 7fb707400700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb6ec0a3cc0 0x7fb6ec0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009fd0 tx=0x7fb6f800b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:41.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.054+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6f001f020 con 0x7fb708103450 2024-04-26T01:57:41.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.285+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb70802df80 con 0x7fb708103450 2024-04-26T01:57:41.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.287+0000 7fb704c00700 1 -- 172.21.15.37:0/2510223338 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22499 (secure 0 0 0) 0x7fb6f0022350 con 0x7fb708103450 2024-04-26T01:57:41.287 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:57:41.287 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":91,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:54:44.157091+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:57:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb6ec0a3cc0 msgr2=0x7fb6ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb6ec0a3cc0 0x7fb6ec0a6180 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009fd0 tx=0x7fb6f800b700 comp rx=0 tx=0).stop 2024-04-26T01:57:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 msgr2=0x7fb70819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 0x7fb70819b870 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6f000d980 tx=0x7fb6f000dc90 comp rx=0 tx=0).stop 2024-04-26T01:57:41.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 shutdown_connections 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb708102a90 0x7fb70819b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7fb6ec0a3cc0 0x7fb6ec0a6180 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb708108a50 0x7fb70819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 --2- 172.21.15.37:0/2510223338 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb708103450 0x7fb70819b870 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 >> 172.21.15.37:0/2510223338 conn(0x7fb708079ba0 msgr2=0x7fb70810df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 shutdown_connections 2024-04-26T01:57:41.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:41.290+0000 7fb70e200700 1 -- 172.21.15.37:0/2510223338 wait complete. 2024-04-26T01:57:42.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:54:12.162501+0000', 'modified': '2024-04-26T01:54:33.402956+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/60713297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 60713297}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 60713297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.71:6839/1148187051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1148187051}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1148187051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24514, 'qdb_cluster': [24514]} 2024-04-26T01:57:42.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T01:57:42.191 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:57:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:42 smithi071 ceph-mon[32578]: pgmap v440: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:42 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2510223338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:57:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:42 smithi037 ceph-mon[27906]: pgmap v440: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:42 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2510223338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:57:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:42 smithi045 ceph-mon[32520]: pgmap v440: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:42 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2510223338' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:57:42.659 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:44.328 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:44 smithi037 ceph-mon[27906]: pgmap v441: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:44 smithi071 ceph-mon[32578]: pgmap v441: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:44 smithi045 ceph-mon[32520]: pgmap v441: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T01:57:44.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 -- 172.21.15.37:0/3941915378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410a240 msgr2=0x7f0604115e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 --2- 172.21.15.37:0/3941915378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410a240 0x7f0604115e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f05fc0099c0 tx=0x7f05fc009cd0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 -- 172.21.15.37:0/3941915378 shutdown_connections 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 --2- 172.21.15.37:0/3941915378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410a240 0x7f0604115e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 --2- 172.21.15.37:0/3941915378 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f060410e1a0 0x7f060410e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 --2- 172.21.15.37:0/3941915378 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0604109880 0x7f0604109d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 -- 172.21.15.37:0/3941915378 >> 172.21.15.37:0/3941915378 conn(0x7f0604078910 msgr2=0x7f060407ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 -- 172.21.15.37:0/3941915378 shutdown_connections 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.891+0000 7f060a600700 1 -- 172.21.15.37:0/3941915378 wait complete. 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.892+0000 7f060a600700 1 Processor -- start 2024-04-26T01:57:44.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.892+0000 7f060a600700 1 -- start start 2024-04-26T01:57:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f060a600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 0x7f060419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f060a600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f060410a240 0x7f060419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f060a600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410e1a0 0x7f06041a3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f060a600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0604119250 con 0x7f060410e1a0 2024-04-26T01:57:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f060a600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f06041190d0 con 0x7f060410a240 2024-04-26T01:57:44.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f060a600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f06041193d0 con 0x7f0604109880 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f0603e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410e1a0 0x7f06041a3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f0603400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 0x7f060419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f0602a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f060410a240 0x7f060419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f0603400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 0x7f060419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44216/0 (socket says 172.21.15.37:44216) 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.893+0000 7f0603400700 1 -- 172.21.15.37:0/1558244772 learned_addr learned my addr 172.21.15.37:0/1558244772 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0602a00700 1 -- 172.21.15.37:0/1558244772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 msgr2=0x7f060419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0602a00700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 0x7f060419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0602a00700 1 -- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410e1a0 msgr2=0x7f06041a3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0602a00700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410e1a0 0x7f06041a3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0602a00700 1 -- 172.21.15.37:0/1558244772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06041a4660 con 0x7f060410a240 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0603400700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 0x7f060419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f0602a00700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f060410a240 0x7f060419fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05f000e9f0 tx=0x7f05f000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f05fbe00700 1 -- 172.21.15.37:0/1558244772 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f000c9c0 con 0x7f060410a240 2024-04-26T01:57:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.894+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06041a4950 con 0x7f060410a240 2024-04-26T01:57:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.895+0000 7f05fbe00700 1 -- 172.21.15.37:0/1558244772 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f000cb60 con 0x7f060410a240 2024-04-26T01:57:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.895+0000 7f05fbe00700 1 -- 172.21.15.37:0/1558244772 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f0018680 con 0x7f060410a240 2024-04-26T01:57:44.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.895+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06041ac1d0 con 0x7f060410a240 2024-04-26T01:57:44.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.898+0000 7f05fbe00700 1 -- 172.21.15.37:0/1558244772 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f05f0010430 con 0x7f060410a240 2024-04-26T01:57:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.901+0000 7f05fbe00700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05e80a3c40 0x7f05e80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.901+0000 7f0614fc9b80 1 Processor -- start 2024-04-26T01:57:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.901+0000 7f05fbe00700 1 -- 172.21.15.37:0/1558244772 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f05f00d05a0 con 0x7f060410a240 2024-04-26T01:57:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.901+0000 7f0603400700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05e80a3c40 0x7f05e80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.901+0000 7f0614fc9b80 1 -- start start 2024-04-26T01:57:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc670f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc671a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc671cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6662e40 con 0x55ecc671aab0 2024-04-26T01:57:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ecc66096e0 con 0x55ecc670f2f0 2024-04-26T01:57:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ecc665ee00 con 0x55ecc670fc10 2024-04-26T01:57:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603400700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05e80a3c40 0x7f05e80a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009c10 tx=0x7f05f400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc670f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc671cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc670f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:51464/0 (socket says 172.21.15.37:51464) 2024-04-26T01:57:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603400700 1 -- 172.21.15.37:0/2866795731 learned_addr learned my addr 172.21.15.37:0/2866795731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0602a00700 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc671a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603400700 1 -- 172.21.15.37:0/2866795731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 msgr2=0x55ecc671a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.902+0000 7f0603400700 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc671a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f0603400700 1 -- 172.21.15.37:0/2866795731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 msgr2=0x55ecc671cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f0603400700 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc671cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f0603400700 1 -- 172.21.15.37:0/2866795731 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc65ff630 con 0x55ecc670f2f0 2024-04-26T01:57:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f0603e00700 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc671cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:44.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f0602a00700 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc671a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f0603400700 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc670f6d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05f400e810 tx=0x7f05f4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e414fa21aa132e46 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f05f8c00700 1 -- 172.21.15.37:0/2866795731 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f401d050 con 0x55ecc670f2f0 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f05f8c00700 1 -- 172.21.15.37:0/2866795731 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f4027070 con 0x55ecc670f2f0 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.903+0000 7f05f8c00700 1 -- 172.21.15.37:0/2866795731 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4022bc0 con 0x55ecc670f2f0 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 -- 172.21.15.37:0/2866795731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 msgr2=0x55ecc670f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc670f6d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05f400e810 tx=0x7f05f4014000 comp rx=0 tx=0).stop 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 -- 172.21.15.37:0/2866795731 shutdown_connections 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc671cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc670f6d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/2866795731 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc671a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 -- 172.21.15.37:0/2866795731 >> 172.21.15.37:0/2866795731 conn(0x55ecc663e550 msgr2=0x55ecc670b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 -- 172.21.15.37:0/2866795731 shutdown_connections 2024-04-26T01:57:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.904+0000 7f0614fc9b80 1 -- 172.21.15.37:0/2866795731 wait complete. 2024-04-26T01:57:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.905+0000 7f0614fc9b80 1 -- start start 2024-04-26T01:57:44.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.905+0000 7f0614fc9b80 1 Processor -- start 2024-04-26T01:57:44.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc671a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc67174b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc67179f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ecc671d5a0 con 0x55ecc671aab0 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ecc661c810 con 0x55ecc670f2f0 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ecc671d720 con 0x55ecc670fc10 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0614fc9b80 10 client.0 _ll_register_callbacks cb 0x55ecc65fb6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc671a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0602a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc67174b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc671a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:51472/0 (socket says 172.21.15.37:51472) 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 -- 172.21.15.37:0/1552168635 learned_addr learned my addr 172.21.15.37:0/1552168635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:44.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603e00700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc67179f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 -- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 msgr2=0x55ecc67174b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc67174b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 -- 172.21.15.37:0/1552168635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 msgr2=0x55ecc67179f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc67179f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.906+0000 7f0603400700 1 -- 172.21.15.37:0/1552168635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc65b0880 con 0x55ecc670f2f0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f0602a00700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc67174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f0603400700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc671a1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f05f40095b0 tx=0x7f05f4009330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20437203d6d763e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f05ed600700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f05ed600700 1 -- 172.21.15.37:0/1552168635 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f4023b10 con 0x55ecc670f2f0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f05ed600700 1 -- 172.21.15.37:0/1552168635 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f05f4023cb0 con 0x55ecc670f2f0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f4027070 con 0x55ecc670f2f0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f05ed600700 1 -- 172.21.15.37:0/1552168635 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f05f4023eb0 con 0x55ecc670f2f0 2024-04-26T01:57:44.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f0614fc9b80 10 client.24923 fetch_fsmap learned FSMap version 19 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.907+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ecc65f9aa0 con 0x55ecc670f2f0 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f05ed600700 1 -- 172.21.15.37:0/1552168635 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f4027070 con 0x55ecc670f2f0 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f05ed600700 1 -- 172.21.15.37:0/1552168635 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f05f4005400 con 0x55ecc670f2f0 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f0614fc9b80 10 client.24923 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f0614fc9b80 10 client.24923 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f0614fc9b80 20 client.24923 populate_metadata read hostname 'smithi037' 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc6729ff0 0x55ecc672b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f0614fc9b80 4 client.24923 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.908+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecc6670630 con 0x55ecc6729ff0 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.909+0000 7f0602a00700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc6729ff0 0x55ecc672b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.909+0000 7f0602a00700 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc6729ff0 0x55ecc672b460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=331214c083d938d4 server_cookie=7d4b5b70c7950a6e in_seq=0 out_seq=0 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.909+0000 7f05ed600700 10 client.24923 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:44.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f05ed600700 1 -- 172.21.15.37:0/1552168635 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ecc6670630 con 0x55ecc6729ff0 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f05ed600700 10 client.24923 handle_command_reply: tid=0 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 client.24923 shutdown 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 msgr2=0x55ecc671a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc671a1b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f05f40095b0 tx=0x7f05f4009330 comp rx=0 tx=0).stop 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 shutdown_connections 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc6729ff0 0x55ecc672b460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc671aab0 0x55ecc67179f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc671a1b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/1552168635 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc670fc10 0x55ecc67174b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f05ed600700 0 client.24923 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.910+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 >> 172.21.15.37:0/1552168635 conn(0x55ecc663e550 msgr2=0x55ecc6711c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.911+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 shutdown_connections 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.911+0000 7f0614fc9b80 1 -- 172.21.15.37:0/1552168635 wait complete. 2024-04-26T01:57:44.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.911+0000 7f0614fc9b80 20 client.24923 trim_cache size 0 max 16384 2024-04-26T01:57:44.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.934+0000 7f0614fc9b80 1 Processor -- start 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.934+0000 7f0614fc9b80 1 -- start start 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6717290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc67177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc67348c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ecc67297a0 con 0x55ecc670fc10 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ecc6729e70 con 0x55ecc670f2f0 2024-04-26T01:57:44.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ecc672a540 con 0x55ecc671aab0 2024-04-26T01:57:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0602a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc67177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0602a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc67177d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49916/0 (socket says 172.21.15.37:49916) 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0602a00700 1 -- 172.21.15.37:0/4234248094 learned_addr learned my addr 172.21.15.37:0/4234248094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.935+0000 7f0603400700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6717290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0603e00700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc67348c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0602a00700 1 -- 172.21.15.37:0/4234248094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 msgr2=0x55ecc67348c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0602a00700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc67348c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0602a00700 1 -- 172.21.15.37:0/4234248094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 msgr2=0x55ecc6717290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0602a00700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6717290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0602a00700 1 -- 172.21.15.37:0/4234248094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecc670cad0 con 0x55ecc670fc10 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0603e00700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc67348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0603400700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6717290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f0602a00700 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc67177d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05f00ce050 tx=0x7f05f00cdf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d84aed62dd41680 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f05efe00700 1 -- 172.21.15.37:0/4234248094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00ba070 con 0x55ecc670fc10 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f05efe00700 1 -- 172.21.15.37:0/4234248094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f00a54a0 con 0x55ecc670fc10 2024-04-26T01:57:44.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.936+0000 7f05efe00700 1 -- 172.21.15.37:0/4234248094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f00a5270 con 0x55ecc670fc10 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 -- 172.21.15.37:0/4234248094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 msgr2=0x55ecc67177d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc67177d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f05f00ce050 tx=0x7f05f00cdf20 comp rx=0 tx=0).stop 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 -- 172.21.15.37:0/4234248094 shutdown_connections 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc67348c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6717290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/4234248094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc67177d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.937+0000 7f0614fc9b80 1 -- 172.21.15.37:0/4234248094 >> 172.21.15.37:0/4234248094 conn(0x55ecc663e550 msgr2=0x55ecc6729050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.938+0000 7f0614fc9b80 1 -- 172.21.15.37:0/4234248094 shutdown_connections 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.938+0000 7f0614fc9b80 1 -- 172.21.15.37:0/4234248094 wait complete. 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.938+0000 7f0614fc9b80 1 -- start start 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.938+0000 7f0614fc9b80 1 Processor -- start 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6714330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc6714870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc6714db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ecc672b2e0 con 0x55ecc670fc10 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ecc672ac10 con 0x55ecc670f2f0 2024-04-26T01:57:44.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ecc672b9b0 con 0x55ecc671aab0 2024-04-26T01:57:44.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0614fc9b80 10 client.0 _ll_register_callbacks cb 0x55ecc65fb6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:57:44.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0603e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc6714db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0603e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc6714db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44270/0 (socket says 172.21.15.37:44270) 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0603e00700 1 -- 172.21.15.37:0/753704297 learned_addr learned my addr 172.21.15.37:0/753704297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0602a00700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc6714870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.939+0000 7f0603400700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6714330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0603e00700 1 -- 172.21.15.37:0/753704297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 msgr2=0x55ecc6714330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0603e00700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6714330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0603e00700 1 -- 172.21.15.37:0/753704297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 msgr2=0x55ecc6714870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0603e00700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc6714870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0603e00700 1 -- 172.21.15.37:0/753704297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecc6732df0 con 0x55ecc671aab0 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0603e00700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc6714db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05fc003760 tx=0x7f05fc0052e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbce8f1b7cf33e48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f05ed600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:57:44.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f05ed600700 1 -- 172.21.15.37:0/753704297 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05fc029070 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f00a54a0 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.940+0000 7f05ed600700 1 -- 172.21.15.37:0/753704297 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f05fc00b790 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f05ed600700 1 -- 172.21.15.37:0/753704297 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f05fc035050 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f0614fc9b80 10 client.24967 fetch_fsmap learned FSMap version 19 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ecc67915f0 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f05ed600700 1 -- 172.21.15.37:0/753704297 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05f00a54a0 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f05ed600700 1 -- 172.21.15.37:0/753704297 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f05fc00f7a0 con 0x55ecc671aab0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f0614fc9b80 10 client.24967 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f0614fc9b80 10 client.24967 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f0614fc9b80 20 client.24967 populate_metadata read hostname 'smithi037' 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.941+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc67961e0 0x55ecc6794ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.942+0000 7f0614fc9b80 4 client.24967 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.942+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "status"}) -- 0x55ecc666a6d0 con 0x55ecc67961e0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.942+0000 7f0603400700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc67961e0 0x55ecc6794ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:44.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.942+0000 7f0603400700 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc67961e0 0x55ecc6794ef0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f463bd1db1f68f server_cookie=248eee6c3552d804 in_seq=0 out_seq=0 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.942+0000 7f05ed600700 10 client.24967 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f05ed600700 1 -- 172.21.15.37:0/753704297 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ecc666a6d0 con 0x55ecc67961e0 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f05ed600700 10 client.24967 handle_command_reply: tid=0 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 client.24967 shutdown 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 msgr2=0x55ecc6714db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc6714db0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f05fc003760 tx=0x7f05fc0052e0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 shutdown_connections 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ecc67961e0 0x55ecc6794ef0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ecc671aab0 0x55ecc6714db0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ecc670f2f0 0x55ecc6714330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 --2- 172.21.15.37:0/753704297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ecc670fc10 0x55ecc6714870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 >> 172.21.15.37:0/753704297 conn(0x55ecc663e550 msgr2=0x55ecc6717290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f05ed600700 0 client.24967 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 shutdown_connections 2024-04-26T01:57:44.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 1 -- 172.21.15.37:0/753704297 wait complete. 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.943+0000 7f0614fc9b80 20 client.24967 trim_cache size 0 max 16384 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24514, 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 211.92379967100001, 2024-04-26T01:57:44.946 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 19, 2024-04-26T01:57:44.947 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 91, 2024-04-26T01:57:44.947 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T01:57:44.947 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 249.04607221000001 2024-04-26T01:57:44.947 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T01:57:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05e80a3c40 msgr2=0x7f05e80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05e80a3c40 0x7f05e80a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f05f4009c10 tx=0x7f05f400b040 comp rx=0 tx=0).stop 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f060410a240 msgr2=0x7f060419fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f060410a240 0x7f060419fbe0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f05f000e9f0 tx=0x7f05f000ed00 comp rx=0 tx=0).stop 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 shutdown_connections 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f060410e1a0 0x7f06041a3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f05e80a3c40 0x7f05e80a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0604109880 0x7f060419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 --2- 172.21.15.37:0/1558244772 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f060410a240 0x7f060419fbe0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 >> 172.21.15.37:0/1558244772 conn(0x7f0604078910 msgr2=0x7f0604078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 shutdown_connections 2024-04-26T01:57:44.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:44.944+0000 7f060a600700 1 -- 172.21.15.37:0/1558244772 wait complete. 2024-04-26T01:57:45.823 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T01:57:45.833 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T01:57:45.834 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T01:57:45.834 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T01:57:45.834 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T01:57:45.835 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:57:45.835 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T01:57:45.835 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T01:57:46.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.054+0000 7f7f0d200640 1 -- 172.21.15.37:0/363419956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081053d0 msgr2=0x7f7f08105850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.054+0000 7f7f0d200640 1 --2- 172.21.15.37:0/363419956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081053d0 0x7f7f08105850 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0009ca0 tx=0x7f7ef002a8c0 comp rx=0 tx=0).stop 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 -- 172.21.15.37:0/363419956 shutdown_connections 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 --2- 172.21.15.37:0/363419956 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 0x7f7f0810c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 --2- 172.21.15.37:0/363419956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081053d0 0x7f7f08105850 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 --2- 172.21.15.37:0/363419956 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081041e0 0x7f7f08104600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 -- 172.21.15.37:0/363419956 >> 172.21.15.37:0/363419956 conn(0x7f7f080ff770 msgr2=0x7f7f08101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 -- 172.21.15.37:0/363419956 shutdown_connections 2024-04-26T01:57:46.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.055+0000 7f7f0d200640 1 -- 172.21.15.37:0/363419956 wait complete. 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 Processor -- start 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 -- start start 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 0x7f7f0819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081053d0 0x7f7f0819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 0x7f7f081a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810fd70 con 0x7f7f081041e0 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810fbf0 con 0x7f7f08105d90 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.056+0000 7f7f0d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0810fef0 con 0x7f7f081053d0 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 0x7f7f0819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:46.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 0x7f7f0819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49958/0 (socket says 172.21.15.37:49958) 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 -- 172.21.15.37:0/932596646 learned_addr learned my addr 172.21.15.37:0/932596646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05400640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081053d0 0x7f7f0819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f06800640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 0x7f7f081a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 -- 172.21.15.37:0/932596646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081053d0 msgr2=0x7f7f0819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081053d0 0x7f7f0819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 -- 172.21.15.37:0/932596646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 msgr2=0x7f7f081a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 0x7f7f081a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:46.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 -- 172.21.15.37:0/932596646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f081a2020 con 0x7f7f081041e0 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05400640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081053d0 0x7f7f0819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f06800640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 0x7f7f081a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.057+0000 7f7f05e00640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 0x7f7f0819a400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800e9c0 tx=0x7f7ef800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.058+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef800cd80 con 0x7f7f081041e0 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.058+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef80035f0 con 0x7f7f081041e0 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.058+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef8010690 con 0x7f7f081041e0 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.058+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f081a2310 con 0x7f7f081041e0 2024-04-26T01:57:46.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.058+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f081a2830 con 0x7f7f081041e0 2024-04-26T01:57:46.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.059+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0810b500 con 0x7f7f081041e0 2024-04-26T01:57:46.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.061+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7ef8004000 con 0x7f7f081041e0 2024-04-26T01:57:46.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.063+0000 7f7efea00640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7ee80a3c20 0x7f7ee80a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:46.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.063+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f7ef80d0960 con 0x7f7f081041e0 2024-04-26T01:57:46.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.063+0000 7f7f05400640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7ee80a3c20 0x7f7ee80a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:46.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.064+0000 7f7f05400640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7ee80a3c20 0x7f7ee80a60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005e30 tx=0x7f7ef0004a40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:46.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.067+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ef809d2f0 con 0x7f7f081041e0 2024-04-26T01:57:46.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:46.328+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f7f08062b50 con 0x7f7f081041e0 2024-04-26T01:57:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:46 smithi071 ceph-mon[32578]: pgmap v442: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:46 smithi037 ceph-mon[27906]: pgmap v442: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:46 smithi045 ceph-mon[32520]: pgmap v442: 97 pgs: 97 active+clean; 639 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T01:57:47.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.052+0000 7f7efea00640 1 -- 172.21.15.37:0/932596646 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f7ef809cbf0 con 0x7f7f081041e0 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.055+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7ee80a3c20 msgr2=0x7f7ee80a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.055+0000 7f7f0d200640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7ee80a3c20 0x7f7ee80a60e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0005e30 tx=0x7f7ef0004a40 comp rx=0 tx=0).stop 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.055+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 msgr2=0x7f7f0819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.055+0000 7f7f0d200640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 0x7f7f0819a400 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ef800e9c0 tx=0x7f7ef800ee80 comp rx=0 tx=0).stop 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 shutdown_connections 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f7ee80a3c20 0x7f7ee80a60e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f08105d90 0x7f7f081a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f081053d0 0x7f7f0819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 --2- 172.21.15.37:0/932596646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f081041e0 0x7f7f0819a400 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 >> 172.21.15.37:0/932596646 conn(0x7f7f080ff770 msgr2=0x7f7f081086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 shutdown_connections 2024-04-26T01:57:47.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.056+0000 7f7f0d200640 1 -- 172.21.15.37:0/932596646 wait complete. 2024-04-26T01:57:47.086 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T01:57:47.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 -- 172.21.15.37:0/2467356759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65341041e0 msgr2=0x7f6534104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 --2- 172.21.15.37:0/2467356759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65341041e0 0x7f6534104600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6524009900 tx=0x7f652402a410 comp rx=0 tx=0).stop 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 -- 172.21.15.37:0/2467356759 shutdown_connections 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 --2- 172.21.15.37:0/2467356759 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6534105d90 0x7f653410c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 --2- 172.21.15.37:0/2467356759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341053d0 0x7f6534105850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 --2- 172.21.15.37:0/2467356759 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f65341041e0 0x7f6534104600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 -- 172.21.15.37:0/2467356759 >> 172.21.15.37:0/2467356759 conn(0x7f65340ff770 msgr2=0x7f6534101b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 -- 172.21.15.37:0/2467356759 shutdown_connections 2024-04-26T01:57:47.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.305+0000 7f6538c00640 1 -- 172.21.15.37:0/2467356759 wait complete. 2024-04-26T01:57:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.306+0000 7f6538c00640 1 Processor -- start 2024-04-26T01:57:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.306+0000 7f6538c00640 1 -- start start 2024-04-26T01:57:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.306+0000 7f6538c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341041e0 0x7f653419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6538c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65341053d0 0x7f653419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6538c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 0x7f65341a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6538c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f653410fd70 con 0x7f65341041e0 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6538c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f653410fbf0 con 0x7f6534105d90 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6538c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f653410fef0 con 0x7f65341053d0 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6531e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341041e0 0x7f653419a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 0x7f65341a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 0x7f65341a19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:51512/0 (socket says 172.21.15.37:51512) 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 -- 172.21.15.37:0/592304859 learned_addr learned my addr 172.21.15.37:0/592304859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6531400640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65341053d0 0x7f653419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 -- 172.21.15.37:0/592304859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65341053d0 msgr2=0x7f653419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65341053d0 0x7f653419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 -- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341041e0 msgr2=0x7f653419a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341041e0 0x7f653419a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6532800640 1 -- 172.21.15.37:0/592304859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65341a2020 con 0x7f6534105d90 2024-04-26T01:57:47.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.307+0000 7f6531e00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341041e0 0x7f653419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.308+0000 7f6532800640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 0x7f65341a19b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f652c00eb10 tx=0x7f652c00c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.308+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c004b20 con 0x7f6534105d90 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.308+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f652c004cc0 con 0x7f6534105d90 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.308+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f652c010670 con 0x7f6534105d90 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.308+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65341a2310 con 0x7f6534105d90 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.308+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65341a2810 con 0x7f6534105d90 2024-04-26T01:57:47.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.309+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653410b550 con 0x7f6534105d90 2024-04-26T01:57:47.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.311+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f652c005020 con 0x7f6534105d90 2024-04-26T01:57:47.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.312+0000 7f652aa00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 0x7f65180a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.313+0000 7f6531e00640 1 -- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 msgr2=0x7f65180a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.313+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f652c0d0a60 con 0x7f6534105d90 2024-04-26T01:57:47.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.313+0000 7f6531e00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 0x7f65180a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:57:47.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.316+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f652c09d3f0 con 0x7f6534105d90 2024-04-26T01:57:47.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/932596646' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:57:47.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: ignoring --setuser ceph since I am not root 2024-04-26T01:57:47.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: ignoring --setgroup ceph since I am not root 2024-04-26T01:57:47.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-mgr[34432]: -- 172.21.15.71:0/3024490172 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f35eb365a0 con 0x55f35eb0a800 2024-04-26T01:57:47.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:47.234+0000 7f000b169200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:57:47.417 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:47 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/932596646' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:57:47.417 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:47 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: ignoring --setuser ceph since I am not root 2024-04-26T01:57:47.418 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:47 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: ignoring --setgroup ceph since I am not root 2024-04-26T01:57:47.418 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:47 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:47.418+0000 7ff3cb696200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:57:47.433 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/932596646' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T01:57:47.433 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: ignoring --setuser ceph since I am not root 2024-04-26T01:57:47.433 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: ignoring --setgroup ceph since I am not root 2024-04-26T01:57:47.433 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-mgr[33907]: -- 172.21.15.45:0/779758520 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f414d125a0 con 0x55f414ce7000 2024-04-26T01:57:47.433 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:47.243+0000 7fa66e14d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T01:57:47.433 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:47.434+0000 7fa66e14d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:57:47.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.514+0000 7f6531e00640 1 -- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 msgr2=0x7f65180a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:47.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.514+0000 7f6531e00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 0x7f65180a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:57:47.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.526+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f6534062b50 con 0x7f6534105d90 2024-04-26T01:57:47.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.534+0000 7f652aa00640 1 -- 172.21.15.37:0/592304859 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7f652c09ccf0 con 0x7f6534105d90 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 msgr2=0x7f65180a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 0x7f65180a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 msgr2=0x7f65341a19b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 0x7f65341a19b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f652c00eb10 tx=0x7f652c00c2d0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 shutdown_connections 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f65180a3bf0 0x7f65180a60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6534105d90 0x7f65341a19b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f65341053d0 0x7f653419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 --2- 172.21.15.37:0/592304859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f65341041e0 0x7f653419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.536+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 >> 172.21.15.37:0/592304859 conn(0x7f65340ff770 msgr2=0x7f6534108720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:47.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.537+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 shutdown_connections 2024-04-26T01:57:47.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.537+0000 7f6538c00640 1 -- 172.21.15.37:0/592304859 wait complete. 2024-04-26T01:57:47.553 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T01:57:47.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.746+0000 7f021e200640 1 -- 172.21.15.37:0/3591105767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218101970 msgr2=0x7f0218101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.746+0000 7f021e200640 1 --2- 172.21.15.37:0/3591105767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218101970 0x7f0218101df0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0210009ca0 tx=0x7f021002a8c0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 -- 172.21.15.37:0/3591105767 shutdown_connections 2024-04-26T01:57:47.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 --2- 172.21.15.37:0/3591105767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218101970 0x7f0218101df0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 --2- 172.21.15.37:0/3591105767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218100fd0 0x7f0218101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 --2- 172.21.15.37:0/3591105767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0218100670 0x7f0218100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 -- 172.21.15.37:0/3591105767 >> 172.21.15.37:0/3591105767 conn(0x7f02180fc230 msgr2=0x7f02180fe6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 -- 172.21.15.37:0/3591105767 shutdown_connections 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.747+0000 7f021e200640 1 -- 172.21.15.37:0/3591105767 wait complete. 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.748+0000 7f021e200640 1 Processor -- start 2024-04-26T01:57:47.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.748+0000 7f021e200640 1 -- start start 2024-04-26T01:57:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021e200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0218100670 0x7f02181061f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021e200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 0x7f0218106730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021e200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 0x7f0218102d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021e200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f021810fba0 con 0x7f0218100670 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021e200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f021810fa20 con 0x7f0218100fd0 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021e200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f021810fd20 con 0x7f0218101970 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f0217e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 0x7f0218106730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 0x7f0218102d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f0217e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 0x7f0218106730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:51534/0 (socket says 172.21.15.37:51534) 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f0217e00640 1 -- 172.21.15.37:0/189098039 learned_addr learned my addr 172.21.15.37:0/189098039 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021d800640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 0x7f0218102d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44336/0 (socket says 172.21.15.37:44336) 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.749+0000 7f021ce00640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0218100670 0x7f02181061f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0217e00640 1 -- 172.21.15.37:0/189098039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 msgr2=0x7f0218102d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0217e00640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 0x7f0218102d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0217e00640 1 -- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0218100670 msgr2=0x7f02181061f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0217e00640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0218100670 0x7f02181061f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0217e00640 1 -- 172.21.15.37:0/189098039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0218103620 con 0x7f0218100fd0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f021d800640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 0x7f0218102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0217e00640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 0x7f0218106730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f020400e9f0 tx=0x7f020400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f020400cd80 con 0x7f0218100fd0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02040108e0 con 0x7f0218100fd0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0204018980 con 0x7f0218100fd0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02181a65b0 con 0x7f0218100fd0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.750+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02181a69c0 con 0x7f0218100fd0 2024-04-26T01:57:47.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.752+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f021810b510 con 0x7f0218100fd0 2024-04-26T01:57:47.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.754+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f0204010430 con 0x7f0218100fd0 2024-04-26T01:57:47.754 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:47.541+0000 7fa66e14d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:57:47.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.755+0000 7f0215600640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 0x7f02000a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.756+0000 7f021ce00640 1 -- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 msgr2=0x7f02000a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.756+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f02040d0780 con 0x7f0218100fd0 2024-04-26T01:57:47.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.756+0000 7f021ce00640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 0x7f02000a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:57:47.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.760+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f020409cac0 con 0x7f0218100fd0 2024-04-26T01:57:47.820 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:47.418+0000 7f000b169200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:57:47.820 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:47.519+0000 7f000b169200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:57:47.821 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:47.821+0000 7f000b169200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:57:47.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.957+0000 7f021ce00640 1 -- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 msgr2=0x7f02000a60a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:47.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.957+0000 7f021ce00640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 0x7f02000a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:57:47.958 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:47 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:47.672+0000 7ff3cb696200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:57:47.958 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:47 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:47.816+0000 7ff3cb696200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T01:57:47.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.964+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f0218062b50 con 0x7f0218100fd0 2024-04-26T01:57:47.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.973+0000 7f0215600640 1 -- 172.21.15.37:0/189098039 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f02040a0dc0 con 0x7f0218100fd0 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 msgr2=0x7f02000a60a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 0x7f02000a60a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 msgr2=0x7f0218106730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 0x7f0218106730 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f020400e9f0 tx=0x7f020400eeb0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 shutdown_connections 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f02000a3be0 0x7f02000a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0218101970 0x7f0218102d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0218100fd0 0x7f0218106730 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 --2- 172.21.15.37:0/189098039 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0218100670 0x7f02181061f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:47.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.975+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 >> 172.21.15.37:0/189098039 conn(0x7f02180fc230 msgr2=0x7f02180fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:47.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.976+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 shutdown_connections 2024-04-26T01:57:47.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:47.976+0000 7f021e200640 1 -- 172.21.15.37:0/189098039 wait complete. 2024-04-26T01:57:47.991 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T01:57:48.015 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T01:57:48.015 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T01:57:48.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:48.015 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T01:57:48.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:48.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T01:57:48.130 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-04-26T01:57:48.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T01:57:48.134 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-26T01:57:48.134 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T01:57:48.134 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T01:57:48.134 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T01:57:48.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:48 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/932596646' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:57:48.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:48 smithi071 ceph-mon[32578]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T01:57:48.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:48 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' 2024-04-26T01:57:48.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:48 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' 2024-04-26T01:57:48.153 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:47.905+0000 7f000b169200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:57:48.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:48 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/932596646' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:57:48.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:48 smithi037 ceph-mon[27906]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T01:57:48.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:48 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' 2024-04-26T01:57:48.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:48 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' 2024-04-26T01:57:48.254 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:47.861+0000 7fa66e14d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:57:48.254 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:47.948+0000 7fa66e14d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:57:48.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:48 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/932596646' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T01:57:48.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:48 smithi045 ceph-mon[32520]: mgrmap e19: x(active, since 12m), standbys: z, y 2024-04-26T01:57:48.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:48 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' 2024-04-26T01:57:48.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:48 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' 2024-04-26T01:57:48.547 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:57:48.657 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:48.247+0000 7ff3cb696200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T01:57:48.658 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:48.367+0000 7ff3cb696200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T01:57:49.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:49.074+0000 7f000b169200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:57:49.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:49.243+0000 7f000b169200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:57:49.504 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:49.192+0000 7fa66e14d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:57:49.504 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:49.368+0000 7fa66e14d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:57:49.691 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:49.409+0000 7f000b169200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:57:49.842 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:49.544+0000 7fa66e14d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:57:49.980 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:49.692+0000 7f000b169200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:57:49.981 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:49.784+0000 7f000b169200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:57:49.981 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:49.981+0000 7f000b169200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:57:50.146 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:49.843+0000 7fa66e14d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:57:50.146 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:49.939+0000 7fa66e14d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:57:50.147 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:50.148+0000 7fa66e14d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:57:50.393 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:50.015+0000 7ff3cb696200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T01:57:50.393 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:50.256+0000 7ff3cb696200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T01:57:50.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:50.171+0000 7f000b169200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:57:50.504 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:50.348+0000 7fa66e14d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:57:50.708 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:50.481+0000 7ff3cb696200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T01:57:51.019 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:50.894+0000 7ff3cb696200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 -- 172.21.15.37:0/1124765813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee4103c20 msgr2=0x7f4ee4104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/1124765813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee4103c20 0x7f4ee4104000 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009990 tx=0x7f4ed4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 -- 172.21.15.37:0/1124765813 shutdown_connections 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/1124765813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee410d000 0x7f4ee410f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/1124765813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee4103c20 0x7f4ee4104000 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/1124765813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 0x7f4ee410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 -- 172.21.15.37:0/1124765813 >> 172.21.15.37:0/1124765813 conn(0x7f4ee40fd3d0 msgr2=0x7f4ee40ff7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 -- 172.21.15.37:0/1124765813 shutdown_connections 2024-04-26T01:57:51.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.085+0000 7f4ee3e00700 1 -- 172.21.15.37:0/1124765813 wait complete. 2024-04-26T01:57:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 Processor -- start 2024-04-26T01:57:51.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 -- start start 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 0x7f4ee4101880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 0x7f4ee4101dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee410d000 0x7f4ee4102300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4112c10 con 0x7f4ee4104540 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4112a90 con 0x7f4ee4103c20 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.087+0000 7f4ee3e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee4112d90 con 0x7f4ee410d000 2024-04-26T01:57:51.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 0x7f4ee4101880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 0x7f4ee4101880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54158/0 (socket says 172.21.15.37:54158) 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 -- 172.21.15.37:0/4181900101 learned_addr learned my addr 172.21.15.37:0/4181900101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee3400700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee410d000 0x7f4ee4102300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2000700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 0x7f4ee4101dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 -- 172.21.15.37:0/4181900101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee410d000 msgr2=0x7f4ee4102300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee410d000 0x7f4ee4102300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 -- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 msgr2=0x7f4ee4101dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 0x7f4ee4101dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 -- 172.21.15.37:0/4181900101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee41a7630 con 0x7f4ee4103c20 2024-04-26T01:57:51.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2000700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 0x7f4ee4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:57:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.088+0000 7f4ee2a00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 0x7f4ee4101880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed40057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.089+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4027070 con 0x7f4ee4103c20 2024-04-26T01:57:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.089+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ed4022470 con 0x7f4ee4103c20 2024-04-26T01:57:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.089+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee41a78c0 con 0x7f4ee4103c20 2024-04-26T01:57:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.089+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4017650 con 0x7f4ee4103c20 2024-04-26T01:57:51.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.089+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee41a7df0 con 0x7f4ee4103c20 2024-04-26T01:57:51.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.090+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee4069730 con 0x7f4ee4103c20 2024-04-26T01:57:51.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.092+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4ed402a070 con 0x7f4ee4103c20 2024-04-26T01:57:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.093+0000 7f4edb400700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 0x7f4ec80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.093+0000 7f4ee2000700 1 -- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 msgr2=0x7f4ec80a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.093+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f4ed40d1160 con 0x7f4ee4103c20 2024-04-26T01:57:51.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.093+0000 7f4ee2000700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 0x7f4ec80a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:57:51.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.097+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ed4013070 con 0x7f4ee4103c20 2024-04-26T01:57:51.152 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:50.760+0000 7f000b169200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:57:51.254 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:50.971+0000 7fa66e14d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:57:51.290 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:51 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:51.028+0000 7ff3cb696200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T01:57:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.294+0000 7f4ee2000700 1 -- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 msgr2=0x7f4ec80a61d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:51.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.294+0000 7f4ee2000700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 0x7f4ec80a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:57:51.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.377+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4ee402df80 con 0x7f4ee4103c20 2024-04-26T01:57:51.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.378+0000 7f4edb400700 1 -- 172.21.15.37:0/4181900101 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7f4ed409db80 con 0x7f4ee4103c20 2024-04-26T01:57:51.377 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.380+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 msgr2=0x7f4ec80a61d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.380+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 0x7f4ec80a61d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.380+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 msgr2=0x7f4ee4101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.380+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 0x7f4ee4101880 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed40057c0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 shutdown_connections 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f4ec80a3d10 0x7f4ec80a61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4ee410d000 0x7f4ee4102300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4ee4103c20 0x7f4ee4101880 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 --2- 172.21.15.37:0/4181900101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ee4104540 0x7f4ee4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:51.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 >> 172.21.15.37:0/4181900101 conn(0x7f4ee40fd3d0 msgr2=0x7f4ee40fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:51.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 shutdown_connections 2024-04-26T01:57:51.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:51.381+0000 7f4ee3e00700 1 -- 172.21.15.37:0/4181900101 wait complete. 2024-04-26T01:57:51.429 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:51.177+0000 7f000b169200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:57:51.429 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:51.271+0000 7f000b169200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:57:51.561 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:51 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4181900101' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:57:51.562 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:51 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:51.290+0000 7ff3cb696200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T01:57:51.713 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:51 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4181900101' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:57:51.714 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:51.407+0000 7fa66e14d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:57:51.714 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:51.507+0000 7fa66e14d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:57:51.714 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:51.715+0000 7fa66e14d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:57:51.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:51 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4181900101' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T01:57:51.902 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:51.470+0000 7f000b169200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:57:51.958 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:51 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:51.562+0000 7ff3cb696200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T01:57:52.236 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T01:57:52.236 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T01:57:52.236 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-04-26T01:57:52.236 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T01:57:52.236 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T01:57:52.237 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T01:57:52.237 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T01:57:52.237 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T01:57:52.237 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T01:57:52.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:52.237 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-04-26T01:57:52.296 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T01:57:52.296 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T01:57:52.296 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 41866sec preferred_lft 41866sec 2024-04-26T01:57:52.297 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link noprefixroute 2024-04-26T01:57:52.298 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-04-26T01:57:52.298 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T01:57:52.298 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-04-26T01:57:52.298 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-04-26T01:57:52.299 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T01:57:52.299 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T01:57:52.312 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:51.985+0000 7f000b169200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:57:52.313 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:52.074+0000 7f000b169200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:57:52.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:57:52.429 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:52 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:52.398+0000 7ff3cb696200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T01:57:52.603 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:52.257+0000 7fa66e14d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:57:52.603 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:52.352+0000 7fa66e14d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:57:52.649 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:52.313+0000 7f000b169200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:57:52.649 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:52.444+0000 7f000b169200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:57:52.649 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:52.537+0000 7f000b169200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:57:52.649 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:52.649+0000 7f000b169200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:57:52.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:52.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:52.787 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T01:57:52.851 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-04-26T01:57:52.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:52.855 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-04-26T01:57:52.911 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-04-26T01:57:52.911 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T01:57:52.912 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-26T01:57:52.912 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-26T01:57:52.912 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T01:57:52.912 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:57:52.912 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T01:57:52.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-26T01:57:52.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-26T01:57:52.912 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-26T01:57:52.912 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T01:57:52.958 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:52.604+0000 7fa66e14d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:57:52.958 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:52.741+0000 7fa66e14d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:57:52.959 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:52.840+0000 7fa66e14d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:57:52.959 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:52.960+0000 7fa66e14d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:57:53.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:57:53.302 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:52 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:52.994+0000 7ff3cb696200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T01:57:53.302 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:53.134+0000 7ff3cb696200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T01:57:53.402 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:52.945+0000 7f000b169200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:57:53.683 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:53.418+0000 7ff3cb696200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T01:57:53.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:53.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:53.696 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T01:57:53.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:53.746 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-04-26T01:57:53.754 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:53.273+0000 7fa66e14d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:57:53.796 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:57:53.796 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T01:57:53.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T01:57:53.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T01:57:53.797 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T01:57:53.823 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:53.543+0000 7f000b169200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:57:53.823 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z[34428]: 2024-04-26T01:57:53.755+0000 7f000b169200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:57:53.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:57:53.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-mon[27906]: Standby manager daemon z restarted 2024-04-26T01:57:53.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-mon[27906]: Standby manager daemon z started 2024-04-26T01:57:53.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:57:53.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:57:53.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:57:53.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:53 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:57:53.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:53.965 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T01:57:53.966 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T01:57:54.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:57:54.126 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-mon[32520]: Standby manager daemon z restarted 2024-04-26T01:57:54.126 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-mon[32520]: Standby manager daemon z started 2024-04-26T01:57:54.126 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:57:54.126 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:57:54.127 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:57:54.127 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:57:54.127 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:53 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:53.904+0000 7fa66e14d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:57:54.127 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y[33903]: 2024-04-26T01:57:54.127+0000 7fa66e14d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:57:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-mon[32578]: Standby manager daemon z restarted 2024-04-26T01:57:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-mon[32578]: Standby manager daemon z started 2024-04-26T01:57:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T01:57:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:57:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T01:57:54.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:53 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.71:0/4119721483' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:57:54.197 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:54.112+0000 7ff3cb696200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T01:57:54.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:54.236 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T01:57:54.236 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-04-26T01:57:54.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-04-26T01:57:54.236 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T01:57:54.236 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-04-26T01:57:54.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:57:54.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:54.441 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T01:57:54.441 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T01:57:54.457 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:54.250+0000 7ff3cb696200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T01:57:54.488 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T01:57:54.488 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T01:57:54.489 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T01:57:54.571 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T01:57:54.571 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T01:57:54.571 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/ebf00d11-c3a6-4b13-963a-0c3a72a5ed73 2024-04-26T01:57:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:54.571 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/ebf00d11-c3a6-4b13-963a-0c3a72a5ed73 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:57:54.747 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:57:54.747 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-26T01:57:54.877 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:54.599+0000 7ff3cb696200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T01:57:54.877 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:54.790+0000 7ff3cb696200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: Standby manager daemon y restarted 2024-04-26T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: Standby manager daemon y started 2024-04-26T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:57:54.878 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:57:54.879 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-mon[27906]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: Standby manager daemon y restarted 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: Standby manager daemon y started 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:57:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:54 smithi071 ceph-mon[32578]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:57:55.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:54.932+0000 7ff3cb696200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T01:57:55.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:55 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:55.095+0000 7ff3cb696200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T01:57:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: mgrmap e20: x(active, since 12m), standbys: y, z 2024-04-26T01:57:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: Standby manager daemon y restarted 2024-04-26T01:57:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: Standby manager daemon y started 2024-04-26T01:57:55.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T01:57:55.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T01:57:55.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T01:57:55.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:54 smithi045 ceph-mon[32520]: from='mgr.? 172.21.15.45:0/3185721613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T01:57:55.497 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-26T01:57:55.498 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-26T01:57:55.498 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.45:3300,172.21.15.71:3300" 2024-04-26T01:57:55.498 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 0@5d371fa8-036e-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/ebf00d11-c3a6-4b13-963a-0c3a72a5ed73 2024-04-26T01:57:55.498 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.37:3300/172.21.15.45:3300/172.21.15.71:3300" will pass to kernel 2024-04-26T01:57:55.501 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T01:57:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-04-26T01:57:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T01:57:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T01:57:55.501 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T01:57:55.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T01:57:55.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T01:57:55.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T01:57:55.581 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:57:55.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:57:55.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:55.695 INFO:teuthology.orchestra.run.smithi037.stdout:client14877 2024-04-26T01:57:55.695 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:57:55.696 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T01:57:55.744 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:57:55.840 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:55 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:55.480+0000 7ff3cb696200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T01:57:55.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:57:55.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:55.892 INFO:teuthology.orchestra.run.smithi037.stdout:client14877 2024-04-26T01:57:55.893 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T01:57:55.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T01:57:55.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:55.996 INFO:teuthology.orchestra.run.smithi037.stdout:client14877 2024-04-26T01:57:55.997 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/5d371fa8-036e-11ef-bc93-c7b262605968.client14877/status') 2024-04-26T01:57:56.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T01:57:56.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:55 smithi037 ceph-mon[27906]: mgrmap e21: x(active, since 12m), standbys: y, z 2024-04-26T01:57:56.130 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.14877 (3)192.168.0.1:0/3840473055 2024-04-26T01:57:56.130 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-04-26T01:57:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-04-26T01:57:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-04-26T01:57:56.130 INFO:teuthology.orchestra.run.smithi037.stderr:70 bytes copied, 7.7394e-05 s, 904 kB/s 2024-04-26T01:57:56.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T01:57:56.142 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T01:57:56.154 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T01:57:56.164 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T01:57:56.164 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T01:57:56.165 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T01:57:56.166 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:55 smithi071 ceph-mon[32578]: mgrmap e21: x(active, since 12m), standbys: y, z 2024-04-26T01:57:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:55 smithi045 ceph-mon[32520]: mgrmap e21: x(active, since 12m), standbys: y, z 2024-04-26T01:57:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.367+0000 7f8cfb000640 1 -- 172.21.15.37:0/694767958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf4107cf0 msgr2=0x7f8cf410a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.367+0000 7f8cfb000640 1 --2- 172.21.15.37:0/694767958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf4107cf0 0x7f8cf410a0e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8cec009900 tx=0x7f8cec02a410 comp rx=0 tx=0).stop 2024-04-26T01:57:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.367+0000 7f8cfb000640 1 -- 172.21.15.37:0/694767958 shutdown_connections 2024-04-26T01:57:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.367+0000 7f8cfb000640 1 --2- 172.21.15.37:0/694767958 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf4107cf0 0x7f8cf410a0e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.368+0000 7f8cfb000640 1 --2- 172.21.15.37:0/694767958 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf41053c0 0x7f8cf41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.368+0000 7f8cfb000640 1 --2- 172.21.15.37:0/694767958 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf4102a90 0x7f8cf4104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.368+0000 7f8cfb000640 1 -- 172.21.15.37:0/694767958 >> 172.21.15.37:0/694767958 conn(0x7f8cf40fc420 msgr2=0x7f8cf40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.368+0000 7f8cfb000640 1 -- 172.21.15.37:0/694767958 shutdown_connections 2024-04-26T01:57:56.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.368+0000 7f8cfb000640 1 -- 172.21.15.37:0/694767958 wait complete. 2024-04-26T01:57:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 Processor -- start 2024-04-26T01:57:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 -- start start 2024-04-26T01:57:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf4102a90 0x7f8cf41981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:56.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf41053c0 0x7f8cf4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf4107cf0 0x7f8cf419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf410db40 con 0x7f8cf41053c0 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf410d9c0 con 0x7f8cf4102a90 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cfb000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf410dcc0 con 0x7f8cf4107cf0 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8ceb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf41053c0 0x7f8cf4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8cf9000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf4107cf0 0x7f8cf419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8ceb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf41053c0 0x7f8cf4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53026/0 (socket says 172.21.15.37:53026) 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.369+0000 7f8ceb400640 1 -- 172.21.15.37:0/2098840369 learned_addr learned my addr 172.21.15.37:0/2098840369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf3e00640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf4102a90 0x7f8cf41981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf9000640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf4102a90 msgr2=0x7f8cf41981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:56.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf9000640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8cf4102a90 0x7f8cf41981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf9000640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf41053c0 msgr2=0x7f8cf4198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf9000640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf41053c0 0x7f8cf4198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf9000640 1 -- 172.21.15.37:0/2098840369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf419fe00 con 0x7f8cf4107cf0 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8ceb400640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8cf41053c0 0x7f8cf4198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf9000640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf4107cf0 0x7f8cf419f790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8cec02a930 tx=0x7f8cec003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.370+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec03f070 con 0x7f8cf4107cf0 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.371+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf41a0090 con 0x7f8cf4107cf0 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.371+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cec031c50 con 0x7f8cf4107cf0 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.371+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec035030 con 0x7f8cf4107cf0 2024-04-26T01:57:56.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.371+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf41a0550 con 0x7f8cf4107cf0 2024-04-26T01:57:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.372+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf40fdf10 con 0x7f8cf4107cf0 2024-04-26T01:57:56.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.374+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 21) ==== 138663+0+0 (secure 0 0 0) 0x7f8cec042070 con 0x7f8cf4107cf0 2024-04-26T01:57:56.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.375+0000 7f8cf1600640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 0x7f8cd80a60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.376+0000 7f8cf3e00640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 msgr2=0x7f8cd80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.376+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8211+0+0 (secure 0 0 0) 0x7f8cec0f1490 con 0x7f8cf4107cf0 2024-04-26T01:57:56.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.376+0000 7f8cf3e00640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 0x7f8cd80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T01:57:56.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.378+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8cec0bd7d0 con 0x7f8cf4107cf0 2024-04-26T01:57:56.458 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.301+0000 7ff3cb696200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T01:57:56.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.577+0000 7f8cf3e00640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 msgr2=0x7f8cd80a60b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/691602839 2024-04-26T01:57:56.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.577+0000 7f8cf3e00640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 0x7f8cd80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T01:57:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.609+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 --> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8cf4062390 con 0x7f8cd80a3bf0 2024-04-26T01:57:56.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.629+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 22) ==== 138331+0+0 (secure 0 0 0) 0x7f8cec0b9e40 con 0x7f8cf4107cf0 2024-04-26T01:57:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.630+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 msgr2=0x7f8cd80a60b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T01:57:56.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:56.630+0000 7f8cf1600640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/691602839,v1:172.21.15.37:6801/691602839] conn(0x7f8cd80a3bf0 0x7f8cd80a60b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:56.941 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.606+0000 7ff3cb696200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T01:57:56.941 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff321a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: Active manager daemon x restarted 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: Activating manager daemon x 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: mgrmap e22: x(active, starting, since 0.0125737s), standbys: y, z 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:57:56.942 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:57:56.943 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:57:56.944 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: Manager daemon x is now available 2024-04-26T01:57:56.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:56.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:57:56.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:57:56.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:57:56.945 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:57:57.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: Active manager daemon x restarted 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: Activating manager daemon x 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: mgrmap e22: x(active, starting, since 0.0125737s), standbys: y, z 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:57:57.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:57:57.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: Manager daemon x is now available 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:57:57.155 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:56 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:57:57.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff321a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff321a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff321a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff321a00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff31de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.208 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff31de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.209 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff31de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.209 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff31de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.209 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:57:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:57:56.942+0000 7ff31de00700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:57:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: Active manager daemon x restarted 2024-04-26T01:57:57.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: Activating manager daemon x 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: mgrmap e22: x(active, starting, since 0.0125737s), standbys: y, z 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T01:57:57.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T01:57:57.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: Manager daemon x is now available 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:57:57.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:56 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T01:57:57.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.637+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mon.1 v2:172.21.15.71:3300/0 8 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7f8cec0eefc0 con 0x7f8cf4107cf0 2024-04-26T01:57:57.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.640+0000 7f8cf1600640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8cd8052820 0x7f8cd8052c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.640+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f8cf4062390 con 0x7f8cd8052820 2024-04-26T01:57:57.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.646+0000 7f8cf3e00640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8cd8052820 0x7f8cd8052c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.650+0000 7f8cf3e00640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8cd8052820 0x7f8cd8052c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40097d0 tx=0x7f8ce4005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.736+0000 7f8cf1600640 1 -- 172.21.15.37:0/2098840369 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8cf4062390 con 0x7f8cd8052820 2024-04-26T01:57:57.734 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8cd8052820 msgr2=0x7f8cd8052c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8cd8052820 0x7f8cd8052c00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40097d0 tx=0x7f8ce4005f90 comp rx=0 tx=0).stop 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf4107cf0 msgr2=0x7f8cf419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 --2- 172.21.15.37:0/2098840369 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8cf4107cf0 0x7f8cf419f790 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8cec02a930 tx=0x7f8cec003b90 comp rx=0 tx=0).stop 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cf9000640 1 -- 172.21.15.37:0/2098840369 reap_dead start 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 shutdown_connections 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 >> 172.21.15.37:0/2098840369 conn(0x7f8cf40fc420 msgr2=0x7f8cf41034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 shutdown_connections 2024-04-26T01:57:57.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.738+0000 7f8cfb000640 1 -- 172.21.15.37:0/2098840369 wait complete. 2024-04-26T01:57:57.754 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T01:57:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 -- 172.21.15.37:0/3774098704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284103b70 msgr2=0x7fa28410a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 --2- 172.21.15.37:0/3774098704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284103b70 0x7fa28410a400 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa27c009ca0 tx=0x7fa27c02a8c0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 -- 172.21.15.37:0/3774098704 shutdown_connections 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 --2- 172.21.15.37:0/3774098704 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284103b70 0x7fa28410a400 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 --2- 172.21.15.37:0/3774098704 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa2841031b0 0x7fa284103630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 --2- 172.21.15.37:0/3774098704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa284101fc0 0x7fa2841023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 -- 172.21.15.37:0/3774098704 >> 172.21.15.37:0/3774098704 conn(0x7fa2840fd570 msgr2=0x7fa2840ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 -- 172.21.15.37:0/3774098704 shutdown_connections 2024-04-26T01:57:57.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.931+0000 7fa289200640 1 -- 172.21.15.37:0/3774098704 wait complete. 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 Processor -- start 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 -- start start 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 0x7fa284198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2841031b0 0x7fa284198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa284103b70 0x7fa28419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410db50 con 0x7fa2841031b0 2024-04-26T01:57:57.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410d9d0 con 0x7fa284103b70 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.932+0000 7fa289200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa28410dcd0 con 0x7fa284101fc0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 0x7fa284198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 0x7fa284198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33524/0 (socket says 172.21.15.37:33524) 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2841031b0 0x7fa284198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 -- 172.21.15.37:0/3548988294 learned_addr learned my addr 172.21.15.37:0/3548988294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa282800640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa284103b70 0x7fa28419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 -- 172.21.15.37:0/3548988294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa284103b70 msgr2=0x7fa28419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa284103b70 0x7fa28419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 -- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2841031b0 msgr2=0x7fa284198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2841031b0 0x7fa284198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 -- 172.21.15.37:0/3548988294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28419fe20 con 0x7fa284101fc0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.933+0000 7fa281e00640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 0x7fa284198200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa27400e9c0 tx=0x7fa27400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.934+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27400cd80 con 0x7fa284101fc0 2024-04-26T01:57:57.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.934+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2841a0110 con 0x7fa284101fc0 2024-04-26T01:57:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.934+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa2740035f0 con 0x7fa284101fc0 2024-04-26T01:57:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.934+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa274010690 con 0x7fa284101fc0 2024-04-26T01:57:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.934+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2841a0660 con 0x7fa284101fc0 2024-04-26T01:57:57.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.935+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa284109330 con 0x7fa284101fc0 2024-04-26T01:57:57.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.937+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fa2740037a0 con 0x7fa284101fc0 2024-04-26T01:57:57.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.938+0000 7fa27aa00640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa2680a3990 0x7fa2680a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:57.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.938+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa2740d0490 con 0x7fa284101fc0 2024-04-26T01:57:57.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.939+0000 7fa281400640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa2680a3990 0x7fa2680a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:57.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.939+0000 7fa281400640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa2680a3990 0x7fa2680a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa284199710 tx=0x7fa270005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:57.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:57.943+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa27409d030 con 0x7fa284101fc0 2024-04-26T01:57:58.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.177+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fa284062430 con 0x7fa2680a3990 2024-04-26T01:57:58.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.302+0000 7fa27aa00640 1 -- 172.21.15.37:0/3548988294 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fa284062430 con 0x7fa2680a3990 2024-04-26T01:57:58.301 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa2680a3990 msgr2=0x7fa2680a5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa2680a3990 0x7fa2680a5e50 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa284199710 tx=0x7fa270005f90 comp rx=0 tx=0).stop 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 msgr2=0x7fa284198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 0x7fa284198200 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa27400e9c0 tx=0x7fa27400ee80 comp rx=0 tx=0).stop 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 shutdown_connections 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa2680a3990 0x7fa2680a5e50 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa284103b70 0x7fa28419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa2841031b0 0x7fa284198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 --2- 172.21.15.37:0/3548988294 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa284101fc0 0x7fa284198200 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.305+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 >> 172.21.15.37:0/3548988294 conn(0x7fa2840fd570 msgr2=0x7fa284106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:58.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.306+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 shutdown_connections 2024-04-26T01:57:58.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.306+0000 7fa289200640 1 -- 172.21.15.37:0/3548988294 wait complete. 2024-04-26T01:57:58.323 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 -- 172.21.15.37:0/4038611914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20077ff0 msgr2=0x7fec20078410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 --2- 172.21.15.37:0/4038611914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20077ff0 0x7fec20078410 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fec14009900 tx=0x7fec1402a3f0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 -- 172.21.15.37:0/4038611914 shutdown_connections 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 --2- 172.21.15.37:0/4038611914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076f10 0x7fec20078d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 --2- 172.21.15.37:0/4038611914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec20076570 0x7fec200769d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 --2- 172.21.15.37:0/4038611914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20077ff0 0x7fec20078410 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 -- 172.21.15.37:0/4038611914 >> 172.21.15.37:0/4038611914 conn(0x7fec200ff580 msgr2=0x7fec201019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 -- 172.21.15.37:0/4038611914 shutdown_connections 2024-04-26T01:57:58.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.517+0000 7fec25e00640 1 -- 172.21.15.37:0/4038611914 wait complete. 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.518+0000 7fec25e00640 1 Processor -- start 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 -- start start 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076570 0x7fec2019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 0x7fec2019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec20077ff0 0x7fec201a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec201140a0 con 0x7fec20077ff0 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fec20113f20 con 0x7fec20076570 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fec20114220 con 0x7fec20076f10 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076570 0x7fec2019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 0x7fec2019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec1fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 0x7fec2019ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33556/0 (socket says 172.21.15.37:33556) 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec24a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076570 0x7fec2019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54324/0 (socket says 172.21.15.37:54324) 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec1fe00640 1 -- 172.21.15.37:0/2645386019 learned_addr learned my addr 172.21.15.37:0/2645386019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:58.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.519+0000 7fec25400640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec20077ff0 0x7fec201a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1fe00640 1 -- 172.21.15.37:0/2645386019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076570 msgr2=0x7fec2019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1fe00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076570 0x7fec2019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1fe00640 1 -- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec20077ff0 msgr2=0x7fec201a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1fe00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec20077ff0 0x7fec201a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1fe00640 1 -- 172.21.15.37:0/2645386019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec201a6340 con 0x7fec20076f10 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1fe00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 0x7fec2019ec60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00d950 tx=0x7fec0c00de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c017070 con 0x7fec20076f10 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec0c009c30 con 0x7fec20076f10 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c00f620 con 0x7fec20076f10 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.520+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec201a6630 con 0x7fec20076f10 2024-04-26T01:57:58.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.521+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201a6b30 con 0x7fec20076f10 2024-04-26T01:57:58.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.522+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec200687f0 con 0x7fec20076f10 2024-04-26T01:57:58.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.524+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 23) ==== 138458+0+0 (secure 0 0 0) 0x7fec0c010460 con 0x7fec20076f10 2024-04-26T01:57:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.526+0000 7fec1d600640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fec080a3a10 0x7fec080a5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.526+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fec0c0cfc30 con 0x7fec20076f10 2024-04-26T01:57:58.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.526+0000 7fec24a00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fec080a3a10 0x7fec080a5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:58.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.527+0000 7fec24a00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fec080a3a10 0x7fec080a5ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec14031500 tx=0x7fec14005a20 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:58.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.532+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec0c009de0 con 0x7fec20076f10 2024-04-26T01:57:58.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.652+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7fec0c0987f0 con 0x7fec20076f10 2024-04-26T01:57:58.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.778+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fec20062390 con 0x7fec080a3a10 2024-04-26T01:57:58.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.796+0000 7fec1d600640 1 -- 172.21.15.37:0/2645386019 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fec20062390 con 0x7fec080a3a10 2024-04-26T01:57:58.794 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T01:57:57", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.798+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fec080a3a10 msgr2=0x7fec080a5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.798+0000 7fec25e00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fec080a3a10 0x7fec080a5ed0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fec14031500 tx=0x7fec14005a20 comp rx=0 tx=0).stop 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.798+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 msgr2=0x7fec2019ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.798+0000 7fec25e00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 0x7fec2019ec60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00d950 tx=0x7fec0c00de10 comp rx=0 tx=0).stop 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 shutdown_connections 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fec080a3a10 0x7fec080a5ed0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec20077ff0 0x7fec201a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fec20076f10 0x7fec2019ec60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 --2- 172.21.15.37:0/2645386019 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fec20076570 0x7fec2019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:58.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 >> 172.21.15.37:0/2645386019 conn(0x7fec200ff580 msgr2=0x7fec201028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:58.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 shutdown_connections 2024-04-26T01:57:58.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:58.799+0000 7fec25e00640 1 -- 172.21.15.37:0/2645386019 wait complete. 2024-04-26T01:57:58.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T01:57:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: [26/Apr/2024:01:57:57] ENGINE Bus STARTING 2024-04-26T01:57:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: [26/Apr/2024:01:57:57] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T01:57:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: mgrmap e23: x(active, since 1.03208s), standbys: y, z 2024-04-26T01:57:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:57:58.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: [26/Apr/2024:01:57:57] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T01:57:58.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: [26/Apr/2024:01:57:57] ENGINE Bus STARTED 2024-04-26T01:57:58.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:58 smithi071 ceph-mon[32578]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: [26/Apr/2024:01:57:57] ENGINE Bus STARTING 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: [26/Apr/2024:01:57:57] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: mgrmap e23: x(active, since 1.03208s), standbys: y, z 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: [26/Apr/2024:01:57:57] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: [26/Apr/2024:01:57:57] ENGINE Bus STARTED 2024-04-26T01:57:58.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:58 smithi037 ceph-mon[27906]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:59.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: [26/Apr/2024:01:57:57] ENGINE Bus STARTING 2024-04-26T01:57:59.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: [26/Apr/2024:01:57:57] ENGINE Serving on http://172.21.15.37:8765 2024-04-26T01:57:59.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: mgrmap e23: x(active, since 1.03208s), standbys: y, z 2024-04-26T01:57:59.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: from='client.25012 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:59.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:57:59.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: [26/Apr/2024:01:57:57] ENGINE Serving on https://172.21.15.37:7150 2024-04-26T01:57:59.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: [26/Apr/2024:01:57:57] ENGINE Bus STARTED 2024-04-26T01:57:59.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:58 smithi045 ceph-mon[32520]: from='client.25051 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 -- 172.21.15.37:0/2179781562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4075f90 msgr2=0x7f10b40743e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 --2- 172.21.15.37:0/2179781562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4075f90 0x7f10b40743e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f10a4009900 tx=0x7f10a402a400 comp rx=0 tx=0).stop 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 -- 172.21.15.37:0/2179781562 shutdown_connections 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 --2- 172.21.15.37:0/2179781562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b4109ea0 0x7f10b410c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 --2- 172.21.15.37:0/2179781562 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b40749b0 0x7f10b4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 --2- 172.21.15.37:0/2179781562 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4075f90 0x7f10b40743e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 -- 172.21.15.37:0/2179781562 >> 172.21.15.37:0/2179781562 conn(0x7f10b40fd570 msgr2=0x7f10b40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 -- 172.21.15.37:0/2179781562 shutdown_connections 2024-04-26T01:57:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.010+0000 7f10b9400640 1 -- 172.21.15.37:0/2179781562 wait complete. 2024-04-26T01:57:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.012+0000 7f10b9400640 1 Processor -- start 2024-04-26T01:57:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.012+0000 7f10b9400640 1 -- start start 2024-04-26T01:57:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.012+0000 7f10b9400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b40749b0 0x7f10b419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.012+0000 7f10b9400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 0x7f10b419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b9400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 0x7f10b41a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b9400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410fd30 con 0x7f10b40749b0 2024-04-26T01:57:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b9400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410fbb0 con 0x7f10b4109ea0 2024-04-26T01:57:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b9400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f10b410feb0 con 0x7f10b4075f90 2024-04-26T01:57:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b40749b0 0x7f10b419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 0x7f10b41a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 0x7f10b419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 0x7f10b41a19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54350/0 (socket says 172.21.15.37:54350) 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b2800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b40749b0 0x7f10b419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53204/0 (socket says 172.21.15.37:53204) 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b1e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 0x7f10b419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:33584/0 (socket says 172.21.15.37:33584) 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 -- 172.21.15.37:0/2058560981 learned_addr learned my addr 172.21.15.37:0/2058560981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 -- 172.21.15.37:0/2058560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 msgr2=0x7f10b419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 0x7f10b419a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 -- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b40749b0 msgr2=0x7f10b419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b40749b0 0x7f10b419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b3200640 1 -- 172.21.15.37:0/2058560981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b41a2060 con 0x7f10b4109ea0 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.013+0000 7f10b1e00640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 0x7f10b419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.014+0000 7f10b3200640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 0x7f10b41a19f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00cad0 tx=0x7f10ac00cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.014+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00ae20 con 0x7f10b4109ea0 2024-04-26T01:57:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.014+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10ac005020 con 0x7f10b4109ea0 2024-04-26T01:57:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.014+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ac00f6b0 con 0x7f10b4109ea0 2024-04-26T01:57:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.014+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b41a2350 con 0x7f10b4109ea0 2024-04-26T01:57:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.014+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b41a28a0 con 0x7f10b4109ea0 2024-04-26T01:57:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.016+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b4108bd0 con 0x7f10b4109ea0 2024-04-26T01:57:59.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.019+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 24) ==== 138766+0+0 (secure 0 0 0) 0x7f10ac0051d0 con 0x7f10b4109ea0 2024-04-26T01:57:59.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.020+0000 7f10ab400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f10980a3d90 0x7f10980a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:57:59.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.020+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f10ac0d07c0 con 0x7f10b4109ea0 2024-04-26T01:57:59.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.021+0000 7f10b2800640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f10980a3d90 0x7f10980a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:57:59.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.022+0000 7f10b2800640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f10980a3d90 0x7f10980a6250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10a4031500 tx=0x7f10a4005a20 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:57:59.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.025+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10ac014030 con 0x7f10b4109ea0 2024-04-26T01:57:59.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.312+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f10b4062390 con 0x7f10980a3d90 2024-04-26T01:57:59.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.322+0000 7f10ab400640 1 -- 172.21.15.37:0/2058560981 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f10b4062390 con 0x7f10980a3d90 2024-04-26T01:57:59.321 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.325+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f10980a3d90 msgr2=0x7f10980a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.325+0000 7f10b9400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f10980a3d90 0x7f10980a6250 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10a4031500 tx=0x7f10a4005a20 comp rx=0 tx=0).stop 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 msgr2=0x7f10b41a19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 0x7f10b41a19f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f10ac00cad0 tx=0x7f10ac00cf90 comp rx=0 tx=0).stop 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 shutdown_connections 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f10980a3d90 0x7f10980a6250 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f10b4109ea0 0x7f10b41a19f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f10b4075f90 0x7f10b419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 --2- 172.21.15.37:0/2058560981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10b40749b0 0x7f10b419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:57:59.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 >> 172.21.15.37:0/2058560981 conn(0x7f10b40fd570 msgr2=0x7f10b4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:57:59.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 shutdown_connections 2024-04-26T01:57:59.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:57:59.326+0000 7f10b9400640 1 -- 172.21.15.37:0/2058560981 wait complete. 2024-04-26T01:57:59.342 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T01:57:59.411 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T01:57:59.421 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T01:57:59.499 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T01:57:59.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T01:57:59.505 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-26T01:57:59.520 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T01:57:59.520 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:57:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:57:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-26T01:57:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:57:59 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.957 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:57:59.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-26T01:57:59.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:57:59.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:57:59 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:57:59.969 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: mgrmap e24: x(active, since 2s), standbys: y, z 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: from='client.25057 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:00.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:57:59 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:00.379 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:58:00.319+0000 7ff2f7200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:58:00.379 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:58:00.319+0000 7ff2f7200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:58:00.379 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:58:00.319+0000 7ff2f7200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:58:00.379 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:58:00.319+0000 7ff2f7200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:58:00.379 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T01:58:00.319+0000 7ff2f7200700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T01:58:01.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:58:01.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-26T01:58:01.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: Unable to set osd_memory_target on smithi071 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-26T01:58:01.154 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:00 smithi071 ceph-mon[32578]: Unable to set osd_memory_target on smithi045 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T01:58:01.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:58:01.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: Unable to set osd_memory_target on smithi071 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.212 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-26T01:58:01.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:00 smithi037 ceph-mon[27906]: Unable to set osd_memory_target on smithi045 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='client.25028 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: Unable to set osd_memory_target on smithi071 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:01.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-26T01:58:01.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:00 smithi045 ceph-mon[32520]: Unable to set osd_memory_target on smithi045 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T01:58:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:02 smithi037 ceph-mon[27906]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:02 smithi037 ceph-mon[27906]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-26T01:58:02.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:02 smithi071 ceph-mon[32578]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:02.344 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:02 smithi071 ceph-mon[32578]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-26T01:58:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:02 smithi045 ceph-mon[32520]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:02 smithi045 ceph-mon[32520]: mgrmap e25: x(active, since 4s), standbys: y, z 2024-04-26T01:58:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 -- 172.21.15.37:0/1537158630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6614108ad0 msgr2=0x7f6614108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 --2- 172.21.15.37:0/1537158630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6614108ad0 0x7f6614108eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6608009990 tx=0x7f6608009ca0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 -- 172.21.15.37:0/1537158630 shutdown_connections 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 --2- 172.21.15.37:0/1537158630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6614108ad0 0x7f6614108eb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 --2- 172.21.15.37:0/1537158630 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f66141019d0 0x7f661410f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 --2- 172.21.15.37:0/1537158630 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 -- 172.21.15.37:0/1537158630 >> 172.21.15.37:0/1537158630 conn(0x7f6614077fb0 msgr2=0x7f66140ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 -- 172.21.15.37:0/1537158630 shutdown_connections 2024-04-26T01:58:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.587+0000 7f661b800700 1 -- 172.21.15.37:0/1537158630 wait complete. 2024-04-26T01:58:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.588+0000 7f661b800700 1 Processor -- start 2024-04-26T01:58:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.588+0000 7f661b800700 1 -- start start 2024-04-26T01:58:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f661b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f661b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66141019d0 0x7f6614199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f661b800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6614108ad0 0x7f661419d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f661b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112c10 con 0x7f66141019d0 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f661b800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112a90 con 0x7f6614108ad0 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f661b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6614112d90 con 0x7f6614101030 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f6618e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f6618e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:38666/0 (socket says 172.21.15.37:38666) 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f6618e00700 1 -- 172.21.15.37:0/2874812999 learned_addr learned my addr 172.21.15.37:0/2874812999 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f6619800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6614108ad0 0x7f661419d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.589+0000 7f6613e00700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66141019d0 0x7f6614199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6618e00700 1 -- 172.21.15.37:0/2874812999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6614108ad0 msgr2=0x7f661419d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6618e00700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6614108ad0 0x7f661419d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6618e00700 1 -- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66141019d0 msgr2=0x7f6614199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6618e00700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66141019d0 0x7f6614199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6618e00700 1 -- 172.21.15.37:0/2874812999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f661419e130 con 0x7f6614101030 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6619800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6614108ad0 0x7f661419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6618e00700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614199120 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6608005910 tx=0x7f660800b670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608027070 con 0x7f6614101030 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6608022470 con 0x7f6614101030 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f661419e3c0 con 0x7f6614101030 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.590+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6608017650 con 0x7f6614101030 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.591+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66141a5ca0 con 0x7f6614101030 2024-04-26T01:58:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.592+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6614104350 con 0x7f6614101030 2024-04-26T01:58:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.595+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f660802a070 con 0x7f6614101030 2024-04-26T01:58:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.596+0000 7f6611600700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f65fc0a3f50 0x7f65fc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.597+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f66080d1780 con 0x7f6614101030 2024-04-26T01:58:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.597+0000 7f6613e00700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f65fc0a3f50 0x7f65fc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:02.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.598+0000 7f6613e00700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f65fc0a3f50 0x7f65fc0a6410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6614108290 tx=0x7f6600009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:02.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.600+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6608013070 con 0x7f6614101030 2024-04-26T01:58:02.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.834+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f661419a480 con 0x7f65fc0a3f50 2024-04-26T01:58:02.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.843+0000 7f6611600700 1 -- 172.21.15.37:0/2874812999 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257665 (secure 0 0 0) 0x7f661419a480 con 0x7f65fc0a3f50 2024-04-26T01:58:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.850+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f65fc0a3f50 msgr2=0x7f65fc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.850+0000 7f661b800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f65fc0a3f50 0x7f65fc0a6410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6614108290 tx=0x7f6600009260 comp rx=0 tx=0).stop 2024-04-26T01:58:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.850+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 msgr2=0x7f6614199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.850+0000 7f661b800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614199120 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6608005910 tx=0x7f660800b670 comp rx=0 tx=0).stop 2024-04-26T01:58:02.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 shutdown_connections 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f65fc0a3f50 0x7f65fc0a6410 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6614108ad0 0x7f661419d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6614101030 0x7f6614199120 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 --2- 172.21.15.37:0/2874812999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f66141019d0 0x7f6614199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 >> 172.21.15.37:0/2874812999 conn(0x7f6614077fb0 msgr2=0x7f66140ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 shutdown_connections 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:02.851+0000 7f661b800700 1 -- 172.21.15.37:0/2874812999 wait complete. 2024-04-26T01:58:02.853 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:58:03.714 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-26T01:58:02.642474+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1185265,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":662,"num_read_kb":485,"num_write":388,"num_write_kb":4348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":791,"ondisk_log_size":791,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":376152,"kb_used_data":14484,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124565672,"statfs":{"total":1151940427776,"available":1151555248128,"internally_reserved":0,"allocated":14831616,"data_stored":8642584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":370193722},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.978764"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768656+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:57:56.768656+0000","last_peered":"2024-04-26T01:57:56.768656+0000","last_clean":"2024-04-26T01:57:56.768656+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:57:56.768656+0000","last_undegraded":"2024-04-26T01:57:56.768656+0000","last_fullsized":"2024-04-26T01:57:56.768656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.519684+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:56:12.519684+0000","last_peered":"2024-04-26T01:56:12.519684+0000","last_clean":"2024-04-26T01:56:12.519684+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:56:12.519684+0000","last_undegraded":"2024-04-26T01:56:12.519684+0000","last_fullsized":"2024-04-26T01:56:12.519684+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528221+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:56:12.528221+0000","last_peered":"2024-04-26T01:56:12.528221+0000","last_clean":"2024-04-26T01:56:12.528221+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:56:12.528221+0000","last_undegraded":"2024-04-26T01:56:12.528221+0000","last_fullsized":"2024-04-26T01:56:12.528221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025842+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:57:57.025842+0000","last_peered":"2024-04-26T01:57:57.025842+0000","last_clean":"2024-04-26T01:57:57.025842+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:57:57.025842+0000","last_undegraded":"2024-04-26T01:57:57.025842+0000","last_fullsized":"2024-04-26T01:57:57.025842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621864+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:57:56.621864+0000","last_peered":"2024-04-26T01:57:56.621864+0000","last_clean":"2024-04-26T01:57:56.621864+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:57:56.621864+0000","last_undegraded":"2024-04-26T01:57:56.621864+0000","last_fullsized":"2024-04-26T01:57:56.621864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620148+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:57:56.620148+0000","last_peered":"2024-04-26T01:57:56.620148+0000","last_clean":"2024-04-26T01:57:56.620148+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:57:56.620148+0000","last_undegraded":"2024-04-26T01:57:56.620148+0000","last_fullsized":"2024-04-26T01:57:56.620148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607210+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:57:57.607210+0000","last_peered":"2024-04-26T01:57:57.607210+0000","last_clean":"2024-04-26T01:57:57.607210+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:57:57.607210+0000","last_undegraded":"2024-04-26T01:57:57.607210+0000","last_fullsized":"2024-04-26T01:57:57.607210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768960+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:57:56.768960+0000","last_peered":"2024-04-26T01:57:56.768960+0000","last_clean":"2024-04-26T01:57:56.768960+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:57:56.768960+0000","last_undegraded":"2024-04-26T01:57:56.768960+0000","last_fullsized":"2024-04-26T01:57:56.768960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950880+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:57:56.950880+0000","last_peered":"2024-04-26T01:57:56.950880+0000","last_clean":"2024-04-26T01:57:56.950880+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:57:56.950880+0000","last_undegraded":"2024-04-26T01:57:56.950880+0000","last_fullsized":"2024-04-26T01:57:56.950880+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620704+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:57:56.620704+0000","last_peered":"2024-04-26T01:57:56.620704+0000","last_clean":"2024-04-26T01:57:56.620704+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:57:56.620704+0000","last_undegraded":"2024-04-26T01:57:56.620704+0000","last_fullsized":"2024-04-26T01:57:56.620704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607626+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:57:57.607626+0000","last_peered":"2024-04-26T01:57:57.607626+0000","last_clean":"2024-04-26T01:57:57.607626+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:57:57.607626+0000","last_undegraded":"2024-04-26T01:57:57.607626+0000","last_fullsized":"2024-04-26T01:57:57.607626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025794+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:57:57.025794+0000","last_peered":"2024-04-26T01:57:57.025794+0000","last_clean":"2024-04-26T01:57:57.025794+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:57:57.025794+0000","last_undegraded":"2024-04-26T01:57:57.025794+0000","last_fullsized":"2024-04-26T01:57:57.025794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.512060+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:56:12.512060+0000","last_peered":"2024-04-26T01:56:12.512060+0000","last_clean":"2024-04-26T01:56:12.512060+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:56:12.512060+0000","last_undegraded":"2024-04-26T01:56:12.512060+0000","last_fullsized":"2024-04-26T01:56:12.512060+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.521178+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:56:12.521178+0000","last_peered":"2024-04-26T01:56:12.521178+0000","last_clean":"2024-04-26T01:56:12.521178+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:56:12.521178+0000","last_undegraded":"2024-04-26T01:56:12.521178+0000","last_fullsized":"2024-04-26T01:56:12.521178+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.522934+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:56:12.522934+0000","last_peered":"2024-04-26T01:56:12.522934+0000","last_clean":"2024-04-26T01:56:12.522934+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:56:12.522934+0000","last_undegraded":"2024-04-26T01:56:12.522934+0000","last_fullsized":"2024-04-26T01:56:12.522934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950380+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:57:56.950380+0000","last_peered":"2024-04-26T01:57:56.950380+0000","last_clean":"2024-04-26T01:57:56.950380+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:57:56.950380+0000","last_undegraded":"2024-04-26T01:57:56.950380+0000","last_fullsized":"2024-04-26T01:57:56.950380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768783+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:57:56.768783+0000","last_peered":"2024-04-26T01:57:56.768783+0000","last_clean":"2024-04-26T01:57:56.768783+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:57:56.768783+0000","last_undegraded":"2024-04-26T01:57:56.768783+0000","last_fullsized":"2024-04-26T01:57:56.768783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619118+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:57:56.619118+0000","last_peered":"2024-04-26T01:57:56.619118+0000","last_clean":"2024-04-26T01:57:56.619118+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:57:56.619118+0000","last_undegraded":"2024-04-26T01:57:56.619118+0000","last_fullsized":"2024-04-26T01:57:56.619118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950205+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:57:56.950205+0000","last_peered":"2024-04-26T01:57:56.950205+0000","last_clean":"2024-04-26T01:57:56.950205+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:57:56.950205+0000","last_undegraded":"2024-04-26T01:57:56.950205+0000","last_fullsized":"2024-04-26T01:57:56.950205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619690+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:57:56.619690+0000","last_peered":"2024-04-26T01:57:56.619690+0000","last_clean":"2024-04-26T01:57:56.619690+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:57:56.619690+0000","last_undegraded":"2024-04-26T01:57:56.619690+0000","last_fullsized":"2024-04-26T01:57:56.619690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769502+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:57:56.769502+0000","last_peered":"2024-04-26T01:57:56.769502+0000","last_clean":"2024-04-26T01:57:56.769502+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:57:56.769502+0000","last_undegraded":"2024-04-26T01:57:56.769502+0000","last_fullsized":"2024-04-26T01:57:56.769502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770044+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:57:56.770044+0000","last_peered":"2024-04-26T01:57:56.770044+0000","last_clean":"2024-04-26T01:57:56.770044+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:57:56.770044+0000","last_undegraded":"2024-04-26T01:57:56.770044+0000","last_fullsized":"2024-04-26T01:57:56.770044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.528040+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:56:12.528040+0000","last_peered":"2024-04-26T01:56:12.528040+0000","last_clean":"2024-04-26T01:56:12.528040+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:56:12.528040+0000","last_undegraded":"2024-04-26T01:56:12.528040+0000","last_fullsized":"2024-04-26T01:56:12.528040+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622427+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:57:56.622427+0000","last_peered":"2024-04-26T01:57:56.622427+0000","last_clean":"2024-04-26T01:57:56.622427+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:57:56.622427+0000","last_undegraded":"2024-04-26T01:57:56.622427+0000","last_fullsized":"2024-04-26T01:57:56.622427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950679+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:57:56.950679+0000","last_peered":"2024-04-26T01:57:56.950679+0000","last_clean":"2024-04-26T01:57:56.950679+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:57:56.950679+0000","last_undegraded":"2024-04-26T01:57:56.950679+0000","last_fullsized":"2024-04-26T01:57:56.950679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622193+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:57:56.622193+0000","last_peered":"2024-04-26T01:57:56.622193+0000","last_clean":"2024-04-26T01:57:56.622193+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:57:56.622193+0000","last_undegraded":"2024-04-26T01:57:56.622193+0000","last_fullsized":"2024-04-26T01:57:56.622193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777417+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:57:56.777417+0000","last_peered":"2024-04-26T01:57:56.777417+0000","last_clean":"2024-04-26T01:57:56.777417+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:57:56.777417+0000","last_undegraded":"2024-04-26T01:57:56.777417+0000","last_fullsized":"2024-04-26T01:57:56.777417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777047+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:57:56.777047+0000","last_peered":"2024-04-26T01:57:56.777047+0000","last_clean":"2024-04-26T01:57:56.777047+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:57:56.777047+0000","last_undegraded":"2024-04-26T01:57:56.777047+0000","last_fullsized":"2024-04-26T01:57:56.777047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621810+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:57:56.621810+0000","last_peered":"2024-04-26T01:57:56.621810+0000","last_clean":"2024-04-26T01:57:56.621810+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:57:56.621810+0000","last_undegraded":"2024-04-26T01:57:56.621810+0000","last_fullsized":"2024-04-26T01:57:56.621810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620119+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:57:56.620119+0000","last_peered":"2024-04-26T01:57:56.620119+0000","last_clean":"2024-04-26T01:57:56.620119+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:57:56.620119+0000","last_undegraded":"2024-04-26T01:57:56.620119+0000","last_fullsized":"2024-04-26T01:57:56.620119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777189+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:57:56.777189+0000","last_peered":"2024-04-26T01:57:56.777189+0000","last_clean":"2024-04-26T01:57:56.777189+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:57:56.777189+0000","last_undegraded":"2024-04-26T01:57:56.777189+0000","last_fullsized":"2024-04-26T01:57:56.777189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376752+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:54:44.376752+0000","last_peered":"2024-04-26T01:54:44.376752+0000","last_clean":"2024-04-26T01:54:44.376752+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:54:44.376752+0000","last_undegraded":"2024-04-26T01:54:44.376752+0000","last_fullsized":"2024-04-26T01:54:44.376752+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768376+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:57:56.768376+0000","last_peered":"2024-04-26T01:57:56.768376+0000","last_clean":"2024-04-26T01:57:56.768376+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:57:56.768376+0000","last_undegraded":"2024-04-26T01:57:56.768376+0000","last_fullsized":"2024-04-26T01:57:56.768376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776972+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:57:56.776972+0000","last_peered":"2024-04-26T01:57:56.776972+0000","last_clean":"2024-04-26T01:57:56.776972+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:57:56.776972+0000","last_undegraded":"2024-04-26T01:57:56.776972+0000","last_fullsized":"2024-04-26T01:57:56.776972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.618330+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:57:56.618330+0000","last_peered":"2024-04-26T01:57:56.618330+0000","last_clean":"2024-04-26T01:57:56.618330+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:57:56.618330+0000","last_undegraded":"2024-04-26T01:57:56.618330+0000","last_fullsized":"2024-04-26T01:57:56.618330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620569+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:57:56.620569+0000","last_peered":"2024-04-26T01:57:56.620569+0000","last_clean":"2024-04-26T01:57:56.620569+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:57:56.620569+0000","last_undegraded":"2024-04-26T01:57:56.620569+0000","last_fullsized":"2024-04-26T01:57:56.620569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950805+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:57:56.950805+0000","last_peered":"2024-04-26T01:57:56.950805+0000","last_clean":"2024-04-26T01:57:56.950805+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:57:56.950805+0000","last_undegraded":"2024-04-26T01:57:56.950805+0000","last_fullsized":"2024-04-26T01:57:56.950805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607591+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:57:57.607591+0000","last_peered":"2024-04-26T01:57:57.607591+0000","last_clean":"2024-04-26T01:57:57.607591+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:57:57.607591+0000","last_undegraded":"2024-04-26T01:57:57.607591+0000","last_fullsized":"2024-04-26T01:57:57.607591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.517463+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:56:12.517463+0000","last_peered":"2024-04-26T01:56:12.517463+0000","last_clean":"2024-04-26T01:56:12.517463+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:56:12.517463+0000","last_undegraded":"2024-04-26T01:56:12.517463+0000","last_fullsized":"2024-04-26T01:56:12.517463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777417+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:57:56.777417+0000","last_peered":"2024-04-26T01:57:56.777417+0000","last_clean":"2024-04-26T01:57:56.777417+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:57:56.777417+0000","last_undegraded":"2024-04-26T01:57:56.777417+0000","last_fullsized":"2024-04-26T01:57:56.777417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950907+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:57:56.950907+0000","last_peered":"2024-04-26T01:57:56.950907+0000","last_clean":"2024-04-26T01:57:56.950907+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:57:56.950907+0000","last_undegraded":"2024-04-26T01:57:56.950907+0000","last_fullsized":"2024-04-26T01:57:56.950907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'36","reported_seq":493,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.949951+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:57:56.949951+0000","last_peered":"2024-04-26T01:57:56.949951+0000","last_clean":"2024-04-26T01:57:56.949951+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:57:56.949951+0000","last_undegraded":"2024-04-26T01:57:56.949951+0000","last_fullsized":"2024-04-26T01:57:56.949951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69049,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":428,"num_read_kb":281,"num_write":36,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622187+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:57:56.622187+0000","last_peered":"2024-04-26T01:57:56.622187+0000","last_clean":"2024-04-26T01:57:56.622187+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:57:56.622187+0000","last_undegraded":"2024-04-26T01:57:56.622187+0000","last_fullsized":"2024-04-26T01:57:56.622187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950167+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:57:56.950167+0000","last_peered":"2024-04-26T01:57:56.950167+0000","last_clean":"2024-04-26T01:57:56.950167+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:57:56.950167+0000","last_undegraded":"2024-04-26T01:57:56.950167+0000","last_fullsized":"2024-04-26T01:57:56.950167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.529156+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:56:12.529156+0000","last_peered":"2024-04-26T01:56:12.529156+0000","last_clean":"2024-04-26T01:56:12.529156+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:56:12.529156+0000","last_undegraded":"2024-04-26T01:56:12.529156+0000","last_fullsized":"2024-04-26T01:56:12.529156+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777048+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:57:56.777048+0000","last_peered":"2024-04-26T01:57:56.777048+0000","last_clean":"2024-04-26T01:57:56.777048+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:57:56.777048+0000","last_undegraded":"2024-04-26T01:57:56.777048+0000","last_fullsized":"2024-04-26T01:57:56.777048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607531+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:57:57.607531+0000","last_peered":"2024-04-26T01:57:57.607531+0000","last_clean":"2024-04-26T01:57:57.607531+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:57:57.607531+0000","last_undegraded":"2024-04-26T01:57:57.607531+0000","last_fullsized":"2024-04-26T01:57:57.607531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025527+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:57:57.025527+0000","last_peered":"2024-04-26T01:57:57.025527+0000","last_clean":"2024-04-26T01:57:57.025527+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:57:57.025527+0000","last_undegraded":"2024-04-26T01:57:57.025527+0000","last_fullsized":"2024-04-26T01:57:57.025527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607131+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:57:57.607131+0000","last_peered":"2024-04-26T01:57:57.607131+0000","last_clean":"2024-04-26T01:57:57.607131+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:57:57.607131+0000","last_undegraded":"2024-04-26T01:57:57.607131+0000","last_fullsized":"2024-04-26T01:57:57.607131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621717+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:57:56.621717+0000","last_peered":"2024-04-26T01:57:56.621717+0000","last_clean":"2024-04-26T01:57:56.621717+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:57:56.621717+0000","last_undegraded":"2024-04-26T01:57:56.621717+0000","last_fullsized":"2024-04-26T01:57:56.621717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776974+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:57:56.776974+0000","last_peered":"2024-04-26T01:57:56.776974+0000","last_clean":"2024-04-26T01:57:56.776974+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:57:56.776974+0000","last_undegraded":"2024-04-26T01:57:56.776974+0000","last_fullsized":"2024-04-26T01:57:56.776974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622339+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:57:56.622339+0000","last_peered":"2024-04-26T01:57:56.622339+0000","last_clean":"2024-04-26T01:57:56.622339+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:57:56.622339+0000","last_undegraded":"2024-04-26T01:57:56.622339+0000","last_fullsized":"2024-04-26T01:57:56.622339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768569+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:57:56.768569+0000","last_peered":"2024-04-26T01:57:56.768569+0000","last_clean":"2024-04-26T01:57:56.768569+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:57:56.768569+0000","last_undegraded":"2024-04-26T01:57:56.768569+0000","last_fullsized":"2024-04-26T01:57:56.768569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619036+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:57:56.619036+0000","last_peered":"2024-04-26T01:57:56.619036+0000","last_clean":"2024-04-26T01:57:56.619036+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:57:56.619036+0000","last_undegraded":"2024-04-26T01:57:56.619036+0000","last_fullsized":"2024-04-26T01:57:56.619036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620424+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:57:56.620424+0000","last_peered":"2024-04-26T01:57:56.620424+0000","last_clean":"2024-04-26T01:57:56.620424+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:57:56.620424+0000","last_undegraded":"2024-04-26T01:57:56.620424+0000","last_fullsized":"2024-04-26T01:57:56.620424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607359+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:57:57.607359+0000","last_peered":"2024-04-26T01:57:57.607359+0000","last_clean":"2024-04-26T01:57:57.607359+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:57:57.607359+0000","last_undegraded":"2024-04-26T01:57:57.607359+0000","last_fullsized":"2024-04-26T01:57:57.607359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620119+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:57:56.620119+0000","last_peered":"2024-04-26T01:57:56.620119+0000","last_clean":"2024-04-26T01:57:56.620119+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:57:56.620119+0000","last_undegraded":"2024-04-26T01:57:56.620119+0000","last_fullsized":"2024-04-26T01:57:56.620119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777211+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:57:56.777211+0000","last_peered":"2024-04-26T01:57:56.777211+0000","last_clean":"2024-04-26T01:57:56.777211+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:57:56.777211+0000","last_undegraded":"2024-04-26T01:57:56.777211+0000","last_fullsized":"2024-04-26T01:57:56.777211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.376367+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:54:44.376367+0000","last_peered":"2024-04-26T01:54:44.376367+0000","last_clean":"2024-04-26T01:54:44.376367+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:54:44.376367+0000","last_undegraded":"2024-04-26T01:54:44.376367+0000","last_fullsized":"2024-04-26T01:54:44.376367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769430+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:57:56.769430+0000","last_peered":"2024-04-26T01:57:56.769430+0000","last_clean":"2024-04-26T01:57:56.769430+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:57:56.769430+0000","last_undegraded":"2024-04-26T01:57:56.769430+0000","last_fullsized":"2024-04-26T01:57:56.769430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025760+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:57:57.025760+0000","last_peered":"2024-04-26T01:57:57.025760+0000","last_clean":"2024-04-26T01:57:57.025760+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:57:57.025760+0000","last_undegraded":"2024-04-26T01:57:57.025760+0000","last_fullsized":"2024-04-26T01:57:57.025760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":361,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607237+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:57:57.607237+0000","last_peered":"2024-04-26T01:57:57.607237+0000","last_clean":"2024-04-26T01:57:57.607237+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:57:57.607237+0000","last_undegraded":"2024-04-26T01:57:57.607237+0000","last_fullsized":"2024-04-26T01:57:57.607237+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.505430+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:56:12.505430+0000","last_peered":"2024-04-26T01:56:12.505430+0000","last_clean":"2024-04-26T01:56:12.505430+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:56:12.505430+0000","last_undegraded":"2024-04-26T01:56:12.505430+0000","last_fullsized":"2024-04-26T01:56:12.505430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607351+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:57:57.607351+0000","last_peered":"2024-04-26T01:57:57.607351+0000","last_clean":"2024-04-26T01:57:57.607351+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:57:57.607351+0000","last_undegraded":"2024-04-26T01:57:57.607351+0000","last_fullsized":"2024-04-26T01:57:57.607351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622296+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:57:56.622296+0000","last_peered":"2024-04-26T01:57:56.622296+0000","last_clean":"2024-04-26T01:57:56.622296+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:57:56.622296+0000","last_undegraded":"2024-04-26T01:57:56.622296+0000","last_fullsized":"2024-04-26T01:57:56.622296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.509587+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:56:12.509587+0000","last_peered":"2024-04-26T01:56:12.509587+0000","last_clean":"2024-04-26T01:56:12.509587+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:56:12.509587+0000","last_undegraded":"2024-04-26T01:56:12.509587+0000","last_fullsized":"2024-04-26T01:56:12.509587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'46","reported_seq":108,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.910040+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:57:56.910040+0000","last_peered":"2024-04-26T01:57:56.910040+0000","last_clean":"2024-04-26T01:57:56.910040+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:57:56.910040+0000","last_undegraded":"2024-04-26T01:57:56.910040+0000","last_fullsized":"2024-04-26T01:57:56.910040+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":83,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770213+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:57:56.770213+0000","last_peered":"2024-04-26T01:57:56.770213+0000","last_clean":"2024-04-26T01:57:56.770213+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:57:56.770213+0000","last_undegraded":"2024-04-26T01:57:56.770213+0000","last_fullsized":"2024-04-26T01:57:56.770213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025788+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:57:57.025788+0000","last_peered":"2024-04-26T01:57:57.025788+0000","last_clean":"2024-04-26T01:57:57.025788+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:57:57.025788+0000","last_undegraded":"2024-04-26T01:57:57.025788+0000","last_fullsized":"2024-04-26T01:57:57.025788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.526655+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:56:12.526655+0000","last_peered":"2024-04-26T01:56:12.526655+0000","last_clean":"2024-04-26T01:56:12.526655+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:56:12.526655+0000","last_undegraded":"2024-04-26T01:56:12.526655+0000","last_fullsized":"2024-04-26T01:56:12.526655+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950387+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:57:56.950387+0000","last_peered":"2024-04-26T01:57:56.950387+0000","last_clean":"2024-04-26T01:57:56.950387+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:57:56.950387+0000","last_undegraded":"2024-04-26T01:57:56.950387+0000","last_fullsized":"2024-04-26T01:57:56.950387+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621727+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:57:56.621727+0000","last_peered":"2024-04-26T01:57:56.621727+0000","last_clean":"2024-04-26T01:57:56.621727+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:57:56.621727+0000","last_undegraded":"2024-04-26T01:57:56.621727+0000","last_fullsized":"2024-04-26T01:57:56.621727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769205+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:57:56.769205+0000","last_peered":"2024-04-26T01:57:56.769205+0000","last_clean":"2024-04-26T01:57:56.769205+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:57:56.769205+0000","last_undegraded":"2024-04-26T01:57:56.769205+0000","last_fullsized":"2024-04-26T01:57:56.769205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768498+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:57:56.768498+0000","last_peered":"2024-04-26T01:57:56.768498+0000","last_clean":"2024-04-26T01:57:56.768498+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:57:56.768498+0000","last_undegraded":"2024-04-26T01:57:56.768498+0000","last_fullsized":"2024-04-26T01:57:56.768498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950186+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:57:56.950186+0000","last_peered":"2024-04-26T01:57:56.950186+0000","last_clean":"2024-04-26T01:57:56.950186+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:57:56.950186+0000","last_undegraded":"2024-04-26T01:57:56.950186+0000","last_fullsized":"2024-04-26T01:57:56.950186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.618702+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:57:56.618702+0000","last_peered":"2024-04-26T01:57:56.618702+0000","last_clean":"2024-04-26T01:57:56.618702+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:57:56.618702+0000","last_undegraded":"2024-04-26T01:57:56.618702+0000","last_fullsized":"2024-04-26T01:57:56.618702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621921+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:57:56.621921+0000","last_peered":"2024-04-26T01:57:56.621921+0000","last_clean":"2024-04-26T01:57:56.621921+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:57:56.621921+0000","last_undegraded":"2024-04-26T01:57:56.621921+0000","last_fullsized":"2024-04-26T01:57:56.621921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607367+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:57:57.607367+0000","last_peered":"2024-04-26T01:57:57.607367+0000","last_clean":"2024-04-26T01:57:57.607367+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:57:57.607367+0000","last_undegraded":"2024-04-26T01:57:57.607367+0000","last_fullsized":"2024-04-26T01:57:57.607367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950192+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:57:56.950192+0000","last_peered":"2024-04-26T01:57:56.950192+0000","last_clean":"2024-04-26T01:57:56.950192+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:57:56.950192+0000","last_undegraded":"2024-04-26T01:57:56.950192+0000","last_fullsized":"2024-04-26T01:57:56.950192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607705+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:57:57.607705+0000","last_peered":"2024-04-26T01:57:57.607705+0000","last_clean":"2024-04-26T01:57:57.607705+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:57:57.607705+0000","last_undegraded":"2024-04-26T01:57:57.607705+0000","last_fullsized":"2024-04-26T01:57:57.607705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'39","reported_seq":76,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.903370+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:57:56.903370+0000","last_peered":"2024-04-26T01:57:56.903370+0000","last_clean":"2024-04-26T01:57:56.903370+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:57:56.903370+0000","last_undegraded":"2024-04-26T01:57:56.903370+0000","last_fullsized":"2024-04-26T01:57:56.903370+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607490+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:57:57.607490+0000","last_peered":"2024-04-26T01:57:57.607490+0000","last_clean":"2024-04-26T01:57:57.607490+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:57:57.607490+0000","last_undegraded":"2024-04-26T01:57:57.607490+0000","last_fullsized":"2024-04-26T01:57:57.607490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:54:44.377311+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:54:44.377311+0000","last_peered":"2024-04-26T01:54:44.377311+0000","last_clean":"2024-04-26T01:54:44.377311+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:54:44.377311+0000","last_undegraded":"2024-04-26T01:54:44.377311+0000","last_fullsized":"2024-04-26T01:54:44.377311+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607764+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:57:57.607764+0000","last_peered":"2024-04-26T01:57:57.607764+0000","last_clean":"2024-04-26T01:57:57.607764+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:57:57.607764+0000","last_undegraded":"2024-04-26T01:57:57.607764+0000","last_fullsized":"2024-04-26T01:57:57.607764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768557+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:57:56.768557+0000","last_peered":"2024-04-26T01:57:56.768557+0000","last_clean":"2024-04-26T01:57:56.768557+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:57:56.768557+0000","last_undegraded":"2024-04-26T01:57:56.768557+0000","last_fullsized":"2024-04-26T01:57:56.768557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.949978+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:57:56.949978+0000","last_peered":"2024-04-26T01:57:56.949978+0000","last_clean":"2024-04-26T01:57:56.949978+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:57:56.949978+0000","last_undegraded":"2024-04-26T01:57:56.949978+0000","last_fullsized":"2024-04-26T01:57:56.949978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950009+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:57:56.950009+0000","last_peered":"2024-04-26T01:57:56.950009+0000","last_clean":"2024-04-26T01:57:56.950009+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:57:56.950009+0000","last_undegraded":"2024-04-26T01:57:56.950009+0000","last_fullsized":"2024-04-26T01:57:56.950009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.525989+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:56:12.525989+0000","last_peered":"2024-04-26T01:56:12.525989+0000","last_clean":"2024-04-26T01:56:12.525989+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:56:12.525989+0000","last_undegraded":"2024-04-26T01:56:12.525989+0000","last_fullsized":"2024-04-26T01:56:12.525989+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.524789+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:56:12.524789+0000","last_peered":"2024-04-26T01:56:12.524789+0000","last_clean":"2024-04-26T01:56:12.524789+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:56:12.524789+0000","last_undegraded":"2024-04-26T01:56:12.524789+0000","last_fullsized":"2024-04-26T01:56:12.524789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768629+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:57:56.768629+0000","last_peered":"2024-04-26T01:57:56.768629+0000","last_clean":"2024-04-26T01:57:56.768629+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:57:56.768629+0000","last_undegraded":"2024-04-26T01:57:56.768629+0000","last_fullsized":"2024-04-26T01:57:56.768629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950313+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:57:56.950313+0000","last_peered":"2024-04-26T01:57:56.950313+0000","last_clean":"2024-04-26T01:57:56.950313+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:57:56.950313+0000","last_undegraded":"2024-04-26T01:57:56.950313+0000","last_fullsized":"2024-04-26T01:57:56.950313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770087+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:57:56.770087+0000","last_peered":"2024-04-26T01:57:56.770087+0000","last_clean":"2024-04-26T01:57:56.770087+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:57:56.770087+0000","last_undegraded":"2024-04-26T01:57:56.770087+0000","last_fullsized":"2024-04-26T01:57:56.770087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776988+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:57:56.776988+0000","last_peered":"2024-04-26T01:57:56.776988+0000","last_clean":"2024-04-26T01:57:56.776988+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:57:56.776988+0000","last_undegraded":"2024-04-26T01:57:56.776988+0000","last_fullsized":"2024-04-26T01:57:56.776988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769011+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:57:56.769011+0000","last_peered":"2024-04-26T01:57:56.769011+0000","last_clean":"2024-04-26T01:57:56.769011+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:57:56.769011+0000","last_undegraded":"2024-04-26T01:57:56.769011+0000","last_fullsized":"2024-04-26T01:57:56.769011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950624+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:57:56.950624+0000","last_peered":"2024-04-26T01:57:56.950624+0000","last_clean":"2024-04-26T01:57:56.950624+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:57:56.950624+0000","last_undegraded":"2024-04-26T01:57:56.950624+0000","last_fullsized":"2024-04-26T01:57:56.950624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T01:56:12.527056+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:56:12.527056+0000","last_peered":"2024-04-26T01:56:12.527056+0000","last_clean":"2024-04-26T01:56:12.527056+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:56:12.527056+0000","last_undegraded":"2024-04-26T01:56:12.527056+0000","last_fullsized":"2024-04-26T01:56:12.527056+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768462+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:57:56.768462+0000","last_peered":"2024-04-26T01:57:56.768462+0000","last_clean":"2024-04-26T01:57:56.768462+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:57:56.768462+0000","last_undegraded":"2024-04-26T01:57:56.768462+0000","last_fullsized":"2024-04-26T01:57:56.768462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":594663,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":526,"num_read_kb":366,"num_write":205,"num_write_kb":1408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1880064,"data_stored":1783989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152,"ondisk_log_size":152,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351041,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36700,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708452,"statfs":{"total":95995035648,"available":95957454848,"internally_reserved":0,"allocated":1392640,"data_stored":885080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612677,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28668,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716484,"statfs":{"total":95995035648,"available":95965679616,"internally_reserved":0,"allocated":1687552,"data_stored":1150810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776201,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32544,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712608,"statfs":{"total":95995035648,"available":95961710592,"internally_reserved":0,"allocated":1396736,"data_stored":885662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037840,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":492468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299477,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28564,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716588,"statfs":{"total":95995035648,"available":95965786112,"internally_reserved":0,"allocated":1585152,"data_stored":1074004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463003,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":987136,"data_stored":483054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1077248,"data_stored":560913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738492,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":987136,"data_stored":483027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411443,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1081344,"data_stored":560947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149804,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1015808,"data_stored":492202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888167,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":491886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724642,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1609728,"data_stored":1082531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":69083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":524920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":131782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":200843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:58:03.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:58:03.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:03.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:03.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.756 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.756 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.756 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.756 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi037 to 177.5M 2024-04-26T01:58:03.756 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Unable to set osd_memory_target on smithi037 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T01:58:03.756 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:03.757 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:03.757 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:58:03.757 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:58:03.757 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:58:03.757 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:03.757 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:03.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:03.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:03.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:03.758 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:03 smithi045 ceph-mon[32520]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:03.788 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:03.788 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:03.788 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.788 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.788 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi037 to 177.5M 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Unable to set osd_memory_target on smithi037 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:03.789 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:03.790 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:03.790 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:03.790 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:03.790 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:03 smithi037 ceph-mon[27906]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:03.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:03.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:03.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi037 to 177.5M 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Unable to set osd_memory_target on smithi037 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi037:/etc/ceph/ceph.conf 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi045:/etc/ceph/ceph.conf 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi071:/etc/ceph/ceph.conf 2024-04-26T01:58:03.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T01:58:03.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:03.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: from='client.25081 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:03.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:03.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi071:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:03.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:03 smithi071 ceph-mon[32578]: Updating smithi045:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:04.176 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:04 smithi071 ceph-mon[32578]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:04.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:04 smithi037 ceph-mon[27906]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: Updating smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: Updating smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:04 smithi045 ceph-mon[32520]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-04-26T01:58:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:05 smithi037 ceph-mon[27906]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:05 smithi037 ceph-mon[27906]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 169 KiB/s wr, 99 op/s 2024-04-26T01:58:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:05 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:05 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.896 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:05 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:05 smithi071 ceph-mon[32578]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:05 smithi071 ceph-mon[32578]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 169 KiB/s wr, 99 op/s 2024-04-26T01:58:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:05 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:05 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:05.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:05 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:05 smithi045 ceph-mon[32520]: Updating smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T01:58:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:05 smithi045 ceph-mon[32520]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 169 KiB/s wr, 99 op/s 2024-04-26T01:58:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:05 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:05 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:06.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:05 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.404+0000 7f0f98400700 1 -- 172.21.15.37:0/2975154679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 msgr2=0x7f0f9010dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.404+0000 7f0f98400700 1 --2- 172.21.15.37:0/2975154679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f9010dcc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009920 tx=0x7f0f7c009c30 comp rx=0 tx=0).stop 2024-04-26T01:58:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.404+0000 7f0f98400700 1 -- 172.21.15.37:0/2975154679 shutdown_connections 2024-04-26T01:58:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.404+0000 7f0f98400700 1 --2- 172.21.15.37:0/2975154679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 0x7f0f90115f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.405+0000 7f0f98400700 1 --2- 172.21.15.37:0/2975154679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f90109310 0x7f0f901096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.405+0000 7f0f98400700 1 --2- 172.21.15.37:0/2975154679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f9010dcc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.405+0000 7f0f98400700 1 -- 172.21.15.37:0/2975154679 >> 172.21.15.37:0/2975154679 conn(0x7f0f90076210 msgr2=0x7f0f90078630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:06.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.405+0000 7f0f98400700 1 -- 172.21.15.37:0/2975154679 shutdown_connections 2024-04-26T01:58:06.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.405+0000 7f0f98400700 1 -- 172.21.15.37:0/2975154679 wait complete. 2024-04-26T01:58:06.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 Processor -- start 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 -- start start 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f90109310 0x7f0f901a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f901a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 0x7f0f9019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90119210 con 0x7f0f90109cc0 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90119090 con 0x7f0f9010e3a0 2024-04-26T01:58:06.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.406+0000 7f0f98400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0f90119390 con 0x7f0f90109310 2024-04-26T01:58:06.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f901a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f96400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 0x7f0f9019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f901a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58066/0 (socket says 172.21.15.37:58066) 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 -- 172.21.15.37:0/2683930296 learned_addr learned my addr 172.21.15.37:0/2683930296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95a00700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f90109310 0x7f0f901a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 -- 172.21.15.37:0/2683930296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f90109310 msgr2=0x7f0f901a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f90109310 0x7f0f901a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 -- 172.21.15.37:0/2683930296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 msgr2=0x7f0f9019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 0x7f0f9019f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f95000700 1 -- 172.21.15.37:0/2683930296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9019ffa0 con 0x7f0f90109cc0 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.407+0000 7f0f96400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 0x7f0f9019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.408+0000 7f0f95000700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f901a5ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009230 tx=0x7f0f7c00f950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.408+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c027070 con 0x7f0f90109cc0 2024-04-26T01:58:06.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.408+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f7c0055f0 con 0x7f0f90109cc0 2024-04-26T01:58:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.408+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c004930 con 0x7f0f90109cc0 2024-04-26T01:58:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.408+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f901a0230 con 0x7f0f90109cc0 2024-04-26T01:58:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.408+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f901ac350 con 0x7f0f90109cc0 2024-04-26T01:58:06.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.409+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f9010a7a0 con 0x7f0f90109cc0 2024-04-26T01:58:06.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.412+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f0f7c02a070 con 0x7f0f90109cc0 2024-04-26T01:58:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.413+0000 7f0f8e000700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0f780a3f50 0x7f0f780a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.413+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0f7c0d17c0 con 0x7f0f90109cc0 2024-04-26T01:58:06.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.413+0000 7f0f95a00700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0f780a3f50 0x7f0f780a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:06.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.414+0000 7f0f95a00700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0f780a3f50 0x7f0f780a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009750 tx=0x7f0f84009310 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:06.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.416+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f7c031020 con 0x7f0f90109cc0 2024-04-26T01:58:06.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.643+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f901a0ba0 con 0x7f0f780a3f50 2024-04-26T01:58:06.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.668+0000 7f0f8e000700 1 -- 172.21.15.37:0/2683930296 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257719 (secure 0 0 0) 0x7f0f901a0ba0 con 0x7f0f780a3f50 2024-04-26T01:58:06.668 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.673+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0f780a3f50 msgr2=0x7f0f780a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.673+0000 7f0f98400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0f780a3f50 0x7f0f780a6410 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009750 tx=0x7f0f84009310 comp rx=0 tx=0).stop 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.673+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 msgr2=0x7f0f901a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.673+0000 7f0f98400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f901a5ae0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c009230 tx=0x7f0f7c00f950 comp rx=0 tx=0).stop 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 shutdown_connections 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0f780a3f50 0x7f0f780a6410 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0f9010e3a0 0x7f0f9019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0f90109310 0x7f0f901a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 --2- 172.21.15.37:0/2683930296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f90109cc0 0x7f0f901a5ae0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 >> 172.21.15.37:0/2683930296 conn(0x7f0f90076210 msgr2=0x7f0f90077620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 shutdown_connections 2024-04-26T01:58:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:06.674+0000 7f0f98400700 1 -- 172.21.15.37:0/2683930296 wait complete. 2024-04-26T01:58:06.674 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:58:06.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:06 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:06.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:06 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:06.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:06 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:06.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:06 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:06.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:06 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:06.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:06 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:06.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:06 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:06.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:06 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:07.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:06 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:58:07.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:06 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:58:07.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:06 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:58:07.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:06 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:58:07.719 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-26T01:58:06.643611+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1383419,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1150,"num_read_kb":1045,"num_write":592,"num_write_kb":5528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":917,"ondisk_log_size":917,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381832,"kb_used_data":15900,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559992,"statfs":{"total":1151940427776,"available":1151549431808,"internally_reserved":0,"allocated":16281600,"data_stored":10052625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":198154,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":488,"num_read_kb":560,"num_write":204,"num_write_kb":1180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.979901"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768656+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:57:56.768656+0000","last_peered":"2024-04-26T01:57:56.768656+0000","last_clean":"2024-04-26T01:57:56.768656+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:57:56.768656+0000","last_undegraded":"2024-04-26T01:57:56.768656+0000","last_fullsized":"2024-04-26T01:57:56.768656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695849+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:57:57.695849+0000","last_peered":"2024-04-26T01:57:57.695849+0000","last_clean":"2024-04-26T01:57:57.695849+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:57:57.695849+0000","last_undegraded":"2024-04-26T01:57:57.695849+0000","last_fullsized":"2024-04-26T01:57:57.695849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694789+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:57:57.694789+0000","last_peered":"2024-04-26T01:57:57.694789+0000","last_clean":"2024-04-26T01:57:57.694789+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:57:57.694789+0000","last_undegraded":"2024-04-26T01:57:57.694789+0000","last_fullsized":"2024-04-26T01:57:57.694789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025842+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:57:57.025842+0000","last_peered":"2024-04-26T01:57:57.025842+0000","last_clean":"2024-04-26T01:57:57.025842+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:57:57.025842+0000","last_undegraded":"2024-04-26T01:57:57.025842+0000","last_fullsized":"2024-04-26T01:57:57.025842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621864+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:57:56.621864+0000","last_peered":"2024-04-26T01:57:56.621864+0000","last_clean":"2024-04-26T01:57:56.621864+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:57:56.621864+0000","last_undegraded":"2024-04-26T01:57:56.621864+0000","last_fullsized":"2024-04-26T01:57:56.621864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620148+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:57:56.620148+0000","last_peered":"2024-04-26T01:57:56.620148+0000","last_clean":"2024-04-26T01:57:56.620148+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:57:56.620148+0000","last_undegraded":"2024-04-26T01:57:56.620148+0000","last_fullsized":"2024-04-26T01:57:56.620148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607210+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:57:57.607210+0000","last_peered":"2024-04-26T01:57:57.607210+0000","last_clean":"2024-04-26T01:57:57.607210+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:57:57.607210+0000","last_undegraded":"2024-04-26T01:57:57.607210+0000","last_fullsized":"2024-04-26T01:57:57.607210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768960+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:57:56.768960+0000","last_peered":"2024-04-26T01:57:56.768960+0000","last_clean":"2024-04-26T01:57:56.768960+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:57:56.768960+0000","last_undegraded":"2024-04-26T01:57:56.768960+0000","last_fullsized":"2024-04-26T01:57:56.768960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950880+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:57:56.950880+0000","last_peered":"2024-04-26T01:57:56.950880+0000","last_clean":"2024-04-26T01:57:56.950880+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:57:56.950880+0000","last_undegraded":"2024-04-26T01:57:56.950880+0000","last_fullsized":"2024-04-26T01:57:56.950880+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620704+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:57:56.620704+0000","last_peered":"2024-04-26T01:57:56.620704+0000","last_clean":"2024-04-26T01:57:56.620704+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:57:56.620704+0000","last_undegraded":"2024-04-26T01:57:56.620704+0000","last_fullsized":"2024-04-26T01:57:56.620704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607626+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:57:57.607626+0000","last_peered":"2024-04-26T01:57:57.607626+0000","last_clean":"2024-04-26T01:57:57.607626+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:57:57.607626+0000","last_undegraded":"2024-04-26T01:57:57.607626+0000","last_fullsized":"2024-04-26T01:57:57.607626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025794+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:57:57.025794+0000","last_peered":"2024-04-26T01:57:57.025794+0000","last_clean":"2024-04-26T01:57:57.025794+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:57:57.025794+0000","last_undegraded":"2024-04-26T01:57:57.025794+0000","last_fullsized":"2024-04-26T01:57:57.025794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695269+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:57:57.695269+0000","last_peered":"2024-04-26T01:57:57.695269+0000","last_clean":"2024-04-26T01:57:57.695269+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:57:57.695269+0000","last_undegraded":"2024-04-26T01:57:57.695269+0000","last_fullsized":"2024-04-26T01:57:57.695269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694860+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:57:57.694860+0000","last_peered":"2024-04-26T01:57:57.694860+0000","last_clean":"2024-04-26T01:57:57.694860+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:57:57.694860+0000","last_undegraded":"2024-04-26T01:57:57.694860+0000","last_fullsized":"2024-04-26T01:57:57.694860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694908+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:57:57.694908+0000","last_peered":"2024-04-26T01:57:57.694908+0000","last_clean":"2024-04-26T01:57:57.694908+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:57:57.694908+0000","last_undegraded":"2024-04-26T01:57:57.694908+0000","last_fullsized":"2024-04-26T01:57:57.694908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950380+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:57:56.950380+0000","last_peered":"2024-04-26T01:57:56.950380+0000","last_clean":"2024-04-26T01:57:56.950380+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:57:56.950380+0000","last_undegraded":"2024-04-26T01:57:56.950380+0000","last_fullsized":"2024-04-26T01:57:56.950380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768783+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:57:56.768783+0000","last_peered":"2024-04-26T01:57:56.768783+0000","last_clean":"2024-04-26T01:57:56.768783+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:57:56.768783+0000","last_undegraded":"2024-04-26T01:57:56.768783+0000","last_fullsized":"2024-04-26T01:57:56.768783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619118+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:57:56.619118+0000","last_peered":"2024-04-26T01:57:56.619118+0000","last_clean":"2024-04-26T01:57:56.619118+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:57:56.619118+0000","last_undegraded":"2024-04-26T01:57:56.619118+0000","last_fullsized":"2024-04-26T01:57:56.619118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950205+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:57:56.950205+0000","last_peered":"2024-04-26T01:57:56.950205+0000","last_clean":"2024-04-26T01:57:56.950205+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:57:56.950205+0000","last_undegraded":"2024-04-26T01:57:56.950205+0000","last_fullsized":"2024-04-26T01:57:56.950205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619690+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:57:56.619690+0000","last_peered":"2024-04-26T01:57:56.619690+0000","last_clean":"2024-04-26T01:57:56.619690+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:57:56.619690+0000","last_undegraded":"2024-04-26T01:57:56.619690+0000","last_fullsized":"2024-04-26T01:57:56.619690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769502+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:57:56.769502+0000","last_peered":"2024-04-26T01:57:56.769502+0000","last_clean":"2024-04-26T01:57:56.769502+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:57:56.769502+0000","last_undegraded":"2024-04-26T01:57:56.769502+0000","last_fullsized":"2024-04-26T01:57:56.769502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770044+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:57:56.770044+0000","last_peered":"2024-04-26T01:57:56.770044+0000","last_clean":"2024-04-26T01:57:56.770044+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:57:56.770044+0000","last_undegraded":"2024-04-26T01:57:56.770044+0000","last_fullsized":"2024-04-26T01:57:56.770044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.696232+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:57:57.696232+0000","last_peered":"2024-04-26T01:57:57.696232+0000","last_clean":"2024-04-26T01:57:57.696232+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:57:57.696232+0000","last_undegraded":"2024-04-26T01:57:57.696232+0000","last_fullsized":"2024-04-26T01:57:57.696232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622427+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:57:56.622427+0000","last_peered":"2024-04-26T01:57:56.622427+0000","last_clean":"2024-04-26T01:57:56.622427+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:57:56.622427+0000","last_undegraded":"2024-04-26T01:57:56.622427+0000","last_fullsized":"2024-04-26T01:57:56.622427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950679+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:57:56.950679+0000","last_peered":"2024-04-26T01:57:56.950679+0000","last_clean":"2024-04-26T01:57:56.950679+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:57:56.950679+0000","last_undegraded":"2024-04-26T01:57:56.950679+0000","last_fullsized":"2024-04-26T01:57:56.950679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622193+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:57:56.622193+0000","last_peered":"2024-04-26T01:57:56.622193+0000","last_clean":"2024-04-26T01:57:56.622193+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:57:56.622193+0000","last_undegraded":"2024-04-26T01:57:56.622193+0000","last_fullsized":"2024-04-26T01:57:56.622193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777417+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:57:56.777417+0000","last_peered":"2024-04-26T01:57:56.777417+0000","last_clean":"2024-04-26T01:57:56.777417+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:57:56.777417+0000","last_undegraded":"2024-04-26T01:57:56.777417+0000","last_fullsized":"2024-04-26T01:57:56.777417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777047+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:57:56.777047+0000","last_peered":"2024-04-26T01:57:56.777047+0000","last_clean":"2024-04-26T01:57:56.777047+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:57:56.777047+0000","last_undegraded":"2024-04-26T01:57:56.777047+0000","last_fullsized":"2024-04-26T01:57:56.777047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621810+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:57:56.621810+0000","last_peered":"2024-04-26T01:57:56.621810+0000","last_clean":"2024-04-26T01:57:56.621810+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:57:56.621810+0000","last_undegraded":"2024-04-26T01:57:56.621810+0000","last_fullsized":"2024-04-26T01:57:56.621810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620119+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:57:56.620119+0000","last_peered":"2024-04-26T01:57:56.620119+0000","last_clean":"2024-04-26T01:57:56.620119+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:57:56.620119+0000","last_undegraded":"2024-04-26T01:57:56.620119+0000","last_fullsized":"2024-04-26T01:57:56.620119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777189+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:57:56.777189+0000","last_peered":"2024-04-26T01:57:56.777189+0000","last_clean":"2024-04-26T01:57:56.777189+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:57:56.777189+0000","last_undegraded":"2024-04-26T01:57:56.777189+0000","last_fullsized":"2024-04-26T01:57:56.777189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694756+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:57:57.694756+0000","last_peered":"2024-04-26T01:57:57.694756+0000","last_clean":"2024-04-26T01:57:57.694756+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:57:57.694756+0000","last_undegraded":"2024-04-26T01:57:57.694756+0000","last_fullsized":"2024-04-26T01:57:57.694756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768376+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:57:56.768376+0000","last_peered":"2024-04-26T01:57:56.768376+0000","last_clean":"2024-04-26T01:57:56.768376+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:57:56.768376+0000","last_undegraded":"2024-04-26T01:57:56.768376+0000","last_fullsized":"2024-04-26T01:57:56.768376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776972+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:57:56.776972+0000","last_peered":"2024-04-26T01:57:56.776972+0000","last_clean":"2024-04-26T01:57:56.776972+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:57:56.776972+0000","last_undegraded":"2024-04-26T01:57:56.776972+0000","last_fullsized":"2024-04-26T01:57:56.776972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.618330+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:57:56.618330+0000","last_peered":"2024-04-26T01:57:56.618330+0000","last_clean":"2024-04-26T01:57:56.618330+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:57:56.618330+0000","last_undegraded":"2024-04-26T01:57:56.618330+0000","last_fullsized":"2024-04-26T01:57:56.618330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620569+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:57:56.620569+0000","last_peered":"2024-04-26T01:57:56.620569+0000","last_clean":"2024-04-26T01:57:56.620569+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:57:56.620569+0000","last_undegraded":"2024-04-26T01:57:56.620569+0000","last_fullsized":"2024-04-26T01:57:56.620569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950805+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:57:56.950805+0000","last_peered":"2024-04-26T01:57:56.950805+0000","last_clean":"2024-04-26T01:57:56.950805+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:57:56.950805+0000","last_undegraded":"2024-04-26T01:57:56.950805+0000","last_fullsized":"2024-04-26T01:57:56.950805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607591+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:57:57.607591+0000","last_peered":"2024-04-26T01:57:57.607591+0000","last_clean":"2024-04-26T01:57:57.607591+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:57:57.607591+0000","last_undegraded":"2024-04-26T01:57:57.607591+0000","last_fullsized":"2024-04-26T01:57:57.607591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.696427+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:57:57.696427+0000","last_peered":"2024-04-26T01:57:57.696427+0000","last_clean":"2024-04-26T01:57:57.696427+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:57:57.696427+0000","last_undegraded":"2024-04-26T01:57:57.696427+0000","last_fullsized":"2024-04-26T01:57:57.696427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777417+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:57:56.777417+0000","last_peered":"2024-04-26T01:57:56.777417+0000","last_clean":"2024-04-26T01:57:56.777417+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:57:56.777417+0000","last_undegraded":"2024-04-26T01:57:56.777417+0000","last_fullsized":"2024-04-26T01:57:56.777417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950907+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:57:56.950907+0000","last_peered":"2024-04-26T01:57:56.950907+0000","last_clean":"2024-04-26T01:57:56.950907+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:57:56.950907+0000","last_undegraded":"2024-04-26T01:57:56.950907+0000","last_fullsized":"2024-04-26T01:57:56.950907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":510,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:58:01.852259+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:58:01.852259+0000","last_peered":"2024-04-26T01:58:01.852259+0000","last_clean":"2024-04-26T01:58:01.852259+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:58:01.852259+0000","last_undegraded":"2024-04-26T01:58:01.852259+0000","last_fullsized":"2024-04-26T01:58:01.852259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70571,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":441,"num_read_kb":290,"num_write":40,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622187+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:57:56.622187+0000","last_peered":"2024-04-26T01:57:56.622187+0000","last_clean":"2024-04-26T01:57:56.622187+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:57:56.622187+0000","last_undegraded":"2024-04-26T01:57:56.622187+0000","last_fullsized":"2024-04-26T01:57:56.622187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950167+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:57:56.950167+0000","last_peered":"2024-04-26T01:57:56.950167+0000","last_clean":"2024-04-26T01:57:56.950167+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:57:56.950167+0000","last_undegraded":"2024-04-26T01:57:56.950167+0000","last_fullsized":"2024-04-26T01:57:56.950167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694620+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:57:57.694620+0000","last_peered":"2024-04-26T01:57:57.694620+0000","last_clean":"2024-04-26T01:57:57.694620+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:57:57.694620+0000","last_undegraded":"2024-04-26T01:57:57.694620+0000","last_fullsized":"2024-04-26T01:57:57.694620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777048+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:57:56.777048+0000","last_peered":"2024-04-26T01:57:56.777048+0000","last_clean":"2024-04-26T01:57:56.777048+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:57:56.777048+0000","last_undegraded":"2024-04-26T01:57:56.777048+0000","last_fullsized":"2024-04-26T01:57:56.777048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607531+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:57:57.607531+0000","last_peered":"2024-04-26T01:57:57.607531+0000","last_clean":"2024-04-26T01:57:57.607531+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:57:57.607531+0000","last_undegraded":"2024-04-26T01:57:57.607531+0000","last_fullsized":"2024-04-26T01:57:57.607531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025527+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:57:57.025527+0000","last_peered":"2024-04-26T01:57:57.025527+0000","last_clean":"2024-04-26T01:57:57.025527+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:57:57.025527+0000","last_undegraded":"2024-04-26T01:57:57.025527+0000","last_fullsized":"2024-04-26T01:57:57.025527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607131+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:57:57.607131+0000","last_peered":"2024-04-26T01:57:57.607131+0000","last_clean":"2024-04-26T01:57:57.607131+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:57:57.607131+0000","last_undegraded":"2024-04-26T01:57:57.607131+0000","last_fullsized":"2024-04-26T01:57:57.607131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621717+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:57:56.621717+0000","last_peered":"2024-04-26T01:57:56.621717+0000","last_clean":"2024-04-26T01:57:56.621717+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:57:56.621717+0000","last_undegraded":"2024-04-26T01:57:56.621717+0000","last_fullsized":"2024-04-26T01:57:56.621717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776974+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:57:56.776974+0000","last_peered":"2024-04-26T01:57:56.776974+0000","last_clean":"2024-04-26T01:57:56.776974+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:57:56.776974+0000","last_undegraded":"2024-04-26T01:57:56.776974+0000","last_fullsized":"2024-04-26T01:57:56.776974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622339+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:57:56.622339+0000","last_peered":"2024-04-26T01:57:56.622339+0000","last_clean":"2024-04-26T01:57:56.622339+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:57:56.622339+0000","last_undegraded":"2024-04-26T01:57:56.622339+0000","last_fullsized":"2024-04-26T01:57:56.622339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768569+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:57:56.768569+0000","last_peered":"2024-04-26T01:57:56.768569+0000","last_clean":"2024-04-26T01:57:56.768569+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:57:56.768569+0000","last_undegraded":"2024-04-26T01:57:56.768569+0000","last_fullsized":"2024-04-26T01:57:56.768569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619036+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:57:56.619036+0000","last_peered":"2024-04-26T01:57:56.619036+0000","last_clean":"2024-04-26T01:57:56.619036+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:57:56.619036+0000","last_undegraded":"2024-04-26T01:57:56.619036+0000","last_fullsized":"2024-04-26T01:57:56.619036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620424+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:57:56.620424+0000","last_peered":"2024-04-26T01:57:56.620424+0000","last_clean":"2024-04-26T01:57:56.620424+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:57:56.620424+0000","last_undegraded":"2024-04-26T01:57:56.620424+0000","last_fullsized":"2024-04-26T01:57:56.620424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607359+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:57:57.607359+0000","last_peered":"2024-04-26T01:57:57.607359+0000","last_clean":"2024-04-26T01:57:57.607359+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:57:57.607359+0000","last_undegraded":"2024-04-26T01:57:57.607359+0000","last_fullsized":"2024-04-26T01:57:57.607359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620119+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:57:56.620119+0000","last_peered":"2024-04-26T01:57:56.620119+0000","last_clean":"2024-04-26T01:57:56.620119+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:57:56.620119+0000","last_undegraded":"2024-04-26T01:57:56.620119+0000","last_fullsized":"2024-04-26T01:57:56.620119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777211+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:57:56.777211+0000","last_peered":"2024-04-26T01:57:56.777211+0000","last_clean":"2024-04-26T01:57:56.777211+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:57:56.777211+0000","last_undegraded":"2024-04-26T01:57:56.777211+0000","last_fullsized":"2024-04-26T01:57:56.777211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694641+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:57:57.694641+0000","last_peered":"2024-04-26T01:57:57.694641+0000","last_clean":"2024-04-26T01:57:57.694641+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:57:57.694641+0000","last_undegraded":"2024-04-26T01:57:57.694641+0000","last_fullsized":"2024-04-26T01:57:57.694641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769430+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:57:56.769430+0000","last_peered":"2024-04-26T01:57:56.769430+0000","last_clean":"2024-04-26T01:57:56.769430+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:57:56.769430+0000","last_undegraded":"2024-04-26T01:57:56.769430+0000","last_fullsized":"2024-04-26T01:57:56.769430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025760+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:57:57.025760+0000","last_peered":"2024-04-26T01:57:57.025760+0000","last_clean":"2024-04-26T01:57:57.025760+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:57:57.025760+0000","last_undegraded":"2024-04-26T01:57:57.025760+0000","last_fullsized":"2024-04-26T01:57:57.025760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":405,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:58.824770+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:57:58.824770+0000","last_peered":"2024-04-26T01:57:58.824770+0000","last_clean":"2024-04-26T01:57:58.824770+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:57:58.824770+0000","last_undegraded":"2024-04-26T01:57:58.824770+0000","last_fullsized":"2024-04-26T01:57:58.824770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694632+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:57:57.694632+0000","last_peered":"2024-04-26T01:57:57.694632+0000","last_clean":"2024-04-26T01:57:57.694632+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:57:57.694632+0000","last_undegraded":"2024-04-26T01:57:57.694632+0000","last_fullsized":"2024-04-26T01:57:57.694632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607351+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:57:57.607351+0000","last_peered":"2024-04-26T01:57:57.607351+0000","last_clean":"2024-04-26T01:57:57.607351+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:57:57.607351+0000","last_undegraded":"2024-04-26T01:57:57.607351+0000","last_fullsized":"2024-04-26T01:57:57.607351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622296+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:57:56.622296+0000","last_peered":"2024-04-26T01:57:56.622296+0000","last_clean":"2024-04-26T01:57:56.622296+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:57:56.622296+0000","last_undegraded":"2024-04-26T01:57:56.622296+0000","last_fullsized":"2024-04-26T01:57:56.622296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695365+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:57:57.695365+0000","last_peered":"2024-04-26T01:57:57.695365+0000","last_clean":"2024-04-26T01:57:57.695365+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:57:57.695365+0000","last_undegraded":"2024-04-26T01:57:57.695365+0000","last_fullsized":"2024-04-26T01:57:57.695365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'123","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:58:00.401269+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:58:00.401269+0000","last_peered":"2024-04-26T01:58:00.401269+0000","last_clean":"2024-04-26T01:58:00.401269+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:58:00.401269+0000","last_undegraded":"2024-04-26T01:58:00.401269+0000","last_fullsized":"2024-04-26T01:58:00.401269+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":228,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770213+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:57:56.770213+0000","last_peered":"2024-04-26T01:57:56.770213+0000","last_clean":"2024-04-26T01:57:56.770213+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:57:56.770213+0000","last_undegraded":"2024-04-26T01:57:56.770213+0000","last_fullsized":"2024-04-26T01:57:56.770213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025788+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:57:57.025788+0000","last_peered":"2024-04-26T01:57:57.025788+0000","last_clean":"2024-04-26T01:57:57.025788+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:57:57.025788+0000","last_undegraded":"2024-04-26T01:57:57.025788+0000","last_fullsized":"2024-04-26T01:57:57.025788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695999+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:57:57.695999+0000","last_peered":"2024-04-26T01:57:57.695999+0000","last_clean":"2024-04-26T01:57:57.695999+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:57:57.695999+0000","last_undegraded":"2024-04-26T01:57:57.695999+0000","last_fullsized":"2024-04-26T01:57:57.695999+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950387+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:57:56.950387+0000","last_peered":"2024-04-26T01:57:56.950387+0000","last_clean":"2024-04-26T01:57:56.950387+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:57:56.950387+0000","last_undegraded":"2024-04-26T01:57:56.950387+0000","last_fullsized":"2024-04-26T01:57:56.950387+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621727+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:57:56.621727+0000","last_peered":"2024-04-26T01:57:56.621727+0000","last_clean":"2024-04-26T01:57:56.621727+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:57:56.621727+0000","last_undegraded":"2024-04-26T01:57:56.621727+0000","last_fullsized":"2024-04-26T01:57:56.621727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769205+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:57:56.769205+0000","last_peered":"2024-04-26T01:57:56.769205+0000","last_clean":"2024-04-26T01:57:56.769205+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:57:56.769205+0000","last_undegraded":"2024-04-26T01:57:56.769205+0000","last_fullsized":"2024-04-26T01:57:56.769205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768498+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:57:56.768498+0000","last_peered":"2024-04-26T01:57:56.768498+0000","last_clean":"2024-04-26T01:57:56.768498+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:57:56.768498+0000","last_undegraded":"2024-04-26T01:57:56.768498+0000","last_fullsized":"2024-04-26T01:57:56.768498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950186+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:57:56.950186+0000","last_peered":"2024-04-26T01:57:56.950186+0000","last_clean":"2024-04-26T01:57:56.950186+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:57:56.950186+0000","last_undegraded":"2024-04-26T01:57:56.950186+0000","last_fullsized":"2024-04-26T01:57:56.950186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.618702+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:57:56.618702+0000","last_peered":"2024-04-26T01:57:56.618702+0000","last_clean":"2024-04-26T01:57:56.618702+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:57:56.618702+0000","last_undegraded":"2024-04-26T01:57:56.618702+0000","last_fullsized":"2024-04-26T01:57:56.618702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621921+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:57:56.621921+0000","last_peered":"2024-04-26T01:57:56.621921+0000","last_clean":"2024-04-26T01:57:56.621921+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:57:56.621921+0000","last_undegraded":"2024-04-26T01:57:56.621921+0000","last_fullsized":"2024-04-26T01:57:56.621921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607367+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:57:57.607367+0000","last_peered":"2024-04-26T01:57:57.607367+0000","last_clean":"2024-04-26T01:57:57.607367+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:57:57.607367+0000","last_undegraded":"2024-04-26T01:57:57.607367+0000","last_fullsized":"2024-04-26T01:57:57.607367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950192+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:57:56.950192+0000","last_peered":"2024-04-26T01:57:56.950192+0000","last_clean":"2024-04-26T01:57:56.950192+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:57:56.950192+0000","last_undegraded":"2024-04-26T01:57:56.950192+0000","last_fullsized":"2024-04-26T01:57:56.950192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607705+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:57:57.607705+0000","last_peered":"2024-04-26T01:57:57.607705+0000","last_clean":"2024-04-26T01:57:57.607705+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:57:57.607705+0000","last_undegraded":"2024-04-26T01:57:57.607705+0000","last_fullsized":"2024-04-26T01:57:57.607705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'73","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:58:00.395163+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:58:00.395163+0000","last_peered":"2024-04-26T01:58:00.395163+0000","last_clean":"2024-04-26T01:58:00.395163+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:58:00.395163+0000","last_undegraded":"2024-04-26T01:58:00.395163+0000","last_fullsized":"2024-04-26T01:58:00.395163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607490+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:57:57.607490+0000","last_peered":"2024-04-26T01:57:57.607490+0000","last_clean":"2024-04-26T01:57:57.607490+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:57:57.607490+0000","last_undegraded":"2024-04-26T01:57:57.607490+0000","last_fullsized":"2024-04-26T01:57:57.607490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695129+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:57:57.695129+0000","last_peered":"2024-04-26T01:57:57.695129+0000","last_clean":"2024-04-26T01:57:57.695129+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:57:57.695129+0000","last_undegraded":"2024-04-26T01:57:57.695129+0000","last_fullsized":"2024-04-26T01:57:57.695129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607764+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:57:57.607764+0000","last_peered":"2024-04-26T01:57:57.607764+0000","last_clean":"2024-04-26T01:57:57.607764+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:57:57.607764+0000","last_undegraded":"2024-04-26T01:57:57.607764+0000","last_fullsized":"2024-04-26T01:57:57.607764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768557+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:57:56.768557+0000","last_peered":"2024-04-26T01:57:56.768557+0000","last_clean":"2024-04-26T01:57:56.768557+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:57:56.768557+0000","last_undegraded":"2024-04-26T01:57:56.768557+0000","last_fullsized":"2024-04-26T01:57:56.768557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.949978+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:57:56.949978+0000","last_peered":"2024-04-26T01:57:56.949978+0000","last_clean":"2024-04-26T01:57:56.949978+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:57:56.949978+0000","last_undegraded":"2024-04-26T01:57:56.949978+0000","last_fullsized":"2024-04-26T01:57:56.949978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950009+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:57:56.950009+0000","last_peered":"2024-04-26T01:57:56.950009+0000","last_clean":"2024-04-26T01:57:56.950009+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:57:56.950009+0000","last_undegraded":"2024-04-26T01:57:56.950009+0000","last_fullsized":"2024-04-26T01:57:56.950009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695347+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:57:57.695347+0000","last_peered":"2024-04-26T01:57:57.695347+0000","last_clean":"2024-04-26T01:57:57.695347+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:57:57.695347+0000","last_undegraded":"2024-04-26T01:57:57.695347+0000","last_fullsized":"2024-04-26T01:57:57.695347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695519+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:57:57.695519+0000","last_peered":"2024-04-26T01:57:57.695519+0000","last_clean":"2024-04-26T01:57:57.695519+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:57:57.695519+0000","last_undegraded":"2024-04-26T01:57:57.695519+0000","last_fullsized":"2024-04-26T01:57:57.695519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768629+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:57:56.768629+0000","last_peered":"2024-04-26T01:57:56.768629+0000","last_clean":"2024-04-26T01:57:56.768629+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:57:56.768629+0000","last_undegraded":"2024-04-26T01:57:56.768629+0000","last_fullsized":"2024-04-26T01:57:56.768629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950313+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:57:56.950313+0000","last_peered":"2024-04-26T01:57:56.950313+0000","last_clean":"2024-04-26T01:57:56.950313+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:57:56.950313+0000","last_undegraded":"2024-04-26T01:57:56.950313+0000","last_fullsized":"2024-04-26T01:57:56.950313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770087+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:57:56.770087+0000","last_peered":"2024-04-26T01:57:56.770087+0000","last_clean":"2024-04-26T01:57:56.770087+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:57:56.770087+0000","last_undegraded":"2024-04-26T01:57:56.770087+0000","last_fullsized":"2024-04-26T01:57:56.770087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776988+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:57:56.776988+0000","last_peered":"2024-04-26T01:57:56.776988+0000","last_clean":"2024-04-26T01:57:56.776988+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:57:56.776988+0000","last_undegraded":"2024-04-26T01:57:56.776988+0000","last_fullsized":"2024-04-26T01:57:56.776988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769011+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:57:56.769011+0000","last_peered":"2024-04-26T01:57:56.769011+0000","last_clean":"2024-04-26T01:57:56.769011+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:57:56.769011+0000","last_undegraded":"2024-04-26T01:57:56.769011+0000","last_fullsized":"2024-04-26T01:57:56.769011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950624+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:57:56.950624+0000","last_peered":"2024-04-26T01:57:56.950624+0000","last_clean":"2024-04-26T01:57:56.950624+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:57:56.950624+0000","last_undegraded":"2024-04-26T01:57:56.950624+0000","last_fullsized":"2024-04-26T01:57:56.950624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695212+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:57:57.695212+0000","last_peered":"2024-04-26T01:57:57.695212+0000","last_clean":"2024-04-26T01:57:57.695212+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:57:57.695212+0000","last_undegraded":"2024-04-26T01:57:57.695212+0000","last_fullsized":"2024-04-26T01:57:57.695212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768462+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:57:56.768462+0000","last_peered":"2024-04-26T01:57:56.768462+0000","last_clean":"2024-04-26T01:57:56.768462+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:57:56.768462+0000","last_undegraded":"2024-04-26T01:57:56.768462+0000","last_fullsized":"2024-04-26T01:57:56.768462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":792817,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":913,"num_read_kb":715,"num_write":389,"num_write_kb":2588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":2378451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351042,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36696,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708456,"statfs":{"total":95995035648,"available":95957458944,"internally_reserved":0,"allocated":1392640,"data_stored":885080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612678,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29112,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716040,"statfs":{"total":95995035648,"available":95965224960,"internally_reserved":0,"allocated":2023424,"data_stored":1481346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36700,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708452,"statfs":{"total":95995035648,"available":95957454848,"internally_reserved":0,"allocated":1396736,"data_stored":885662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037841,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":492468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299478,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1082931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":491981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":562435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738493,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":491954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411444,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":562469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149805,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37036,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708116,"statfs":{"total":95995035648,"available":95957110784,"internally_reserved":0,"allocated":1740800,"data_stored":1213650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888168,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":491886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724643,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28912,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716240,"statfs":{"total":95995035648,"available":95965429760,"internally_reserved":0,"allocated":1941504,"data_stored":1410763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":733184,"data_stored":721552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":328414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":398997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:58:07.723 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T01:58:07.723 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T01:58:07.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:07 smithi071 ceph-mon[32578]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 77 op/s 2024-04-26T01:58:07.902 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:07 smithi071 ceph-mon[32578]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:07.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:07 smithi037 ceph-mon[27906]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 77 op/s 2024-04-26T01:58:07.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:07 smithi037 ceph-mon[27906]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:08.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:07 smithi045 ceph-mon[32520]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 131 KiB/s wr, 77 op/s 2024-04-26T01:58:08.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:07 smithi045 ceph-mon[32520]: from='client.14955 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:08.221 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:09.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:09 smithi037 ceph-mon[27906]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 108 KiB/s wr, 64 op/s 2024-04-26T01:58:10.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:09 smithi045 ceph-mon[32520]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 108 KiB/s wr, 64 op/s 2024-04-26T01:58:10.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:09 smithi071 ceph-mon[32578]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 108 KiB/s wr, 64 op/s 2024-04-26T01:58:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 -- 172.21.15.37:0/556682030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 msgr2=0x7f7d141133b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 --2- 172.21.15.37:0/556682030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141133b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098f0 tx=0x7f7d00009c00 comp rx=0 tx=0).stop 2024-04-26T01:58:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 -- 172.21.15.37:0/556682030 shutdown_connections 2024-04-26T01:58:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 --2- 172.21.15.37:0/556682030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d14077350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 --2- 172.21.15.37:0/556682030 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d141138f0 0x7f7d14115ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 --2- 172.21.15.37:0/556682030 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141133b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 -- 172.21.15.37:0/556682030 >> 172.21.15.37:0/556682030 conn(0x7f7d14100740 msgr2=0x7f7d14102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 -- 172.21.15.37:0/556682030 shutdown_connections 2024-04-26T01:58:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.761+0000 7f7d19400700 1 -- 172.21.15.37:0/556682030 wait complete. 2024-04-26T01:58:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 Processor -- start 2024-04-26T01:58:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 -- start start 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d141a5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d141138f0 0x7f7d1419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d141192c0 con 0x7f7d14076f70 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14119140 con 0x7f7d141138f0 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.763+0000 7f7d19400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7d14119440 con 0x7f7d14077890 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d141a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d141a5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60730/0 (socket says 172.21.15.37:60730) 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 -- 172.21.15.37:0/272997808 learned_addr learned my addr 172.21.15.37:0/272997808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d11e00700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d13200700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d141138f0 0x7f7d1419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 -- 172.21.15.37:0/272997808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 msgr2=0x7f7d141a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 -- 172.21.15.37:0/272997808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d141138f0 msgr2=0x7f7d1419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:10.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d141138f0 0x7f7d1419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:10.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d12800700 1 -- 172.21.15.37:0/272997808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d141a0000 con 0x7f7d14076f70 2024-04-26T01:58:10.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.764+0000 7f7d11e00700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:10.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.765+0000 7f7d12800700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d141a5650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400ec10 tx=0x7f7d0400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:10.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.765+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400cc80 con 0x7f7d14076f70 2024-04-26T01:58:10.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.765+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d0400ce20 con 0x7f7d14076f70 2024-04-26T01:58:10.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.765+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d141a02f0 con 0x7f7d14076f70 2024-04-26T01:58:10.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.765+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d04010430 con 0x7f7d14076f70 2024-04-26T01:58:10.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.765+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d141ac5d0 con 0x7f7d14076f70 2024-04-26T01:58:10.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.766+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d140786b0 con 0x7f7d14076f70 2024-04-26T01:58:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.768+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f7d04004000 con 0x7f7d14076f70 2024-04-26T01:58:10.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.770+0000 7f7d0b400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7cf80a3ed0 0x7f7cf80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:10.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.770+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7d040d0d40 con 0x7f7d14076f70 2024-04-26T01:58:10.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.770+0000 7f7d11e00700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7cf80a3ed0 0x7f7cf80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:10.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.771+0000 7f7d11e00700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7cf80a3ed0 0x7f7cf80a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098c0 tx=0x7f7d00019040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:10.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:10.775+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d0409d160 con 0x7f7d14076f70 2024-04-26T01:58:11.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.006+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7d141a0c50 con 0x7f7cf80a3ed0 2024-04-26T01:58:11.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.014+0000 7f7d0b400700 1 -- 172.21.15.37:0/272997808 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257697 (secure 0 0 0) 0x7f7d141a0c50 con 0x7f7cf80a3ed0 2024-04-26T01:58:11.016 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7cf80a3ed0 msgr2=0x7f7cf80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7cf80a3ed0 0x7f7cf80a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7d000098c0 tx=0x7f7d00019040 comp rx=0 tx=0).stop 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 msgr2=0x7f7d141a5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d141a5650 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400ec10 tx=0x7f7d0400ef20 comp rx=0 tx=0).stop 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 shutdown_connections 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d14076f70 0x7f7d141a5650 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7cf80a3ed0 0x7f7cf80a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7d141138f0 0x7f7d1419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 --2- 172.21.15.37:0/272997808 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7d14077890 0x7f7d141a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 >> 172.21.15.37:0/272997808 conn(0x7f7d14100740 msgr2=0x7f7d14102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:11.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 shutdown_connections 2024-04-26T01:58:11.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:11.019+0000 7f7d19400700 1 -- 172.21.15.37:0/272997808 wait complete. 2024-04-26T01:58:11.020 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-04-26T01:58:11.868 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-04-26T01:58:10.645456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1383479,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":1161,"num_read_kb":1051,"num_write":593,"num_write_kb":5529,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":918,"ondisk_log_size":918,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":381832,"kb_used_data":15900,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124559992,"statfs":{"total":1151940427776,"available":1151549431808,"internally_reserved":0,"allocated":16281600,"data_stored":10052805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19142,"internal_metadata":374453562},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":198214,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":499,"num_read_kb":566,"num_write":205,"num_write_kb":1181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004110"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768656+0000","last_change":"2024-04-26T01:54:09.005235+0000","last_active":"2024-04-26T01:57:56.768656+0000","last_peered":"2024-04-26T01:57:56.768656+0000","last_clean":"2024-04-26T01:57:56.768656+0000","last_became_active":"2024-04-26T01:54:09.004905+0000","last_became_peered":"2024-04-26T01:54:09.004905+0000","last_unstale":"2024-04-26T01:57:56.768656+0000","last_undegraded":"2024-04-26T01:57:56.768656+0000","last_fullsized":"2024-04-26T01:57:56.768656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:24:29.248330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695849+0000","last_change":"2024-04-26T01:54:09.003572+0000","last_active":"2024-04-26T01:57:57.695849+0000","last_peered":"2024-04-26T01:57:57.695849+0000","last_clean":"2024-04-26T01:57:57.695849+0000","last_became_active":"2024-04-26T01:54:09.003232+0000","last_became_peered":"2024-04-26T01:54:09.003232+0000","last_unstale":"2024-04-26T01:57:57.695849+0000","last_undegraded":"2024-04-26T01:57:57.695849+0000","last_fullsized":"2024-04-26T01:57:57.695849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:54:10.713642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694789+0000","last_change":"2024-04-26T01:54:08.999122+0000","last_active":"2024-04-26T01:57:57.694789+0000","last_peered":"2024-04-26T01:57:57.694789+0000","last_clean":"2024-04-26T01:57:57.694789+0000","last_became_active":"2024-04-26T01:54:08.998779+0000","last_became_peered":"2024-04-26T01:54:08.998779+0000","last_unstale":"2024-04-26T01:57:57.694789+0000","last_undegraded":"2024-04-26T01:57:57.694789+0000","last_fullsized":"2024-04-26T01:57:57.694789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:10:10.215873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025842+0000","last_change":"2024-04-26T01:54:09.000608+0000","last_active":"2024-04-26T01:57:57.025842+0000","last_peered":"2024-04-26T01:57:57.025842+0000","last_clean":"2024-04-26T01:57:57.025842+0000","last_became_active":"2024-04-26T01:54:09.000246+0000","last_became_peered":"2024-04-26T01:54:09.000246+0000","last_unstale":"2024-04-26T01:57:57.025842+0000","last_undegraded":"2024-04-26T01:57:57.025842+0000","last_fullsized":"2024-04-26T01:57:57.025842+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:46:08.622357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621864+0000","last_change":"2024-04-26T01:54:09.015449+0000","last_active":"2024-04-26T01:57:56.621864+0000","last_peered":"2024-04-26T01:57:56.621864+0000","last_clean":"2024-04-26T01:57:56.621864+0000","last_became_active":"2024-04-26T01:54:09.015095+0000","last_became_peered":"2024-04-26T01:54:09.015095+0000","last_unstale":"2024-04-26T01:57:56.621864+0000","last_undegraded":"2024-04-26T01:57:56.621864+0000","last_fullsized":"2024-04-26T01:57:56.621864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:54:18.000530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620148+0000","last_change":"2024-04-26T01:54:09.006674+0000","last_active":"2024-04-26T01:57:56.620148+0000","last_peered":"2024-04-26T01:57:56.620148+0000","last_clean":"2024-04-26T01:57:56.620148+0000","last_became_active":"2024-04-26T01:54:09.006195+0000","last_became_peered":"2024-04-26T01:54:09.006195+0000","last_unstale":"2024-04-26T01:57:56.620148+0000","last_undegraded":"2024-04-26T01:57:56.620148+0000","last_fullsized":"2024-04-26T01:57:56.620148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:04.918787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607210+0000","last_change":"2024-04-26T01:54:09.006460+0000","last_active":"2024-04-26T01:57:57.607210+0000","last_peered":"2024-04-26T01:57:57.607210+0000","last_clean":"2024-04-26T01:57:57.607210+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:57:57.607210+0000","last_undegraded":"2024-04-26T01:57:57.607210+0000","last_fullsized":"2024-04-26T01:57:57.607210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:20:42.682647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768960+0000","last_change":"2024-04-26T01:54:09.004834+0000","last_active":"2024-04-26T01:57:56.768960+0000","last_peered":"2024-04-26T01:57:56.768960+0000","last_clean":"2024-04-26T01:57:56.768960+0000","last_became_active":"2024-04-26T01:54:09.002872+0000","last_became_peered":"2024-04-26T01:54:09.002872+0000","last_unstale":"2024-04-26T01:57:56.768960+0000","last_undegraded":"2024-04-26T01:57:56.768960+0000","last_fullsized":"2024-04-26T01:57:56.768960+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:06.319089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950880+0000","last_change":"2024-04-26T01:54:09.000521+0000","last_active":"2024-04-26T01:57:56.950880+0000","last_peered":"2024-04-26T01:57:56.950880+0000","last_clean":"2024-04-26T01:57:56.950880+0000","last_became_active":"2024-04-26T01:54:09.000161+0000","last_became_peered":"2024-04-26T01:54:09.000161+0000","last_unstale":"2024-04-26T01:57:56.950880+0000","last_undegraded":"2024-04-26T01:57:56.950880+0000","last_fullsized":"2024-04-26T01:57:56.950880+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:31.579369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620704+0000","last_change":"2024-04-26T01:54:08.996804+0000","last_active":"2024-04-26T01:57:56.620704+0000","last_peered":"2024-04-26T01:57:56.620704+0000","last_clean":"2024-04-26T01:57:56.620704+0000","last_became_active":"2024-04-26T01:54:08.996428+0000","last_became_peered":"2024-04-26T01:54:08.996428+0000","last_unstale":"2024-04-26T01:57:56.620704+0000","last_undegraded":"2024-04-26T01:57:56.620704+0000","last_fullsized":"2024-04-26T01:57:56.620704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:40:09.932246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607626+0000","last_change":"2024-04-26T01:54:09.000502+0000","last_active":"2024-04-26T01:57:57.607626+0000","last_peered":"2024-04-26T01:57:57.607626+0000","last_clean":"2024-04-26T01:57:57.607626+0000","last_became_active":"2024-04-26T01:54:08.997885+0000","last_became_peered":"2024-04-26T01:54:08.997885+0000","last_unstale":"2024-04-26T01:57:57.607626+0000","last_undegraded":"2024-04-26T01:57:57.607626+0000","last_fullsized":"2024-04-26T01:57:57.607626+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:51:06.660582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025794+0000","last_change":"2024-04-26T01:54:08.998017+0000","last_active":"2024-04-26T01:57:57.025794+0000","last_peered":"2024-04-26T01:57:57.025794+0000","last_clean":"2024-04-26T01:57:57.025794+0000","last_became_active":"2024-04-26T01:54:08.997636+0000","last_became_peered":"2024-04-26T01:54:08.997636+0000","last_unstale":"2024-04-26T01:57:57.025794+0000","last_undegraded":"2024-04-26T01:57:57.025794+0000","last_fullsized":"2024-04-26T01:57:57.025794+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:52.256347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695269+0000","last_change":"2024-04-26T01:54:09.006018+0000","last_active":"2024-04-26T01:57:57.695269+0000","last_peered":"2024-04-26T01:57:57.695269+0000","last_clean":"2024-04-26T01:57:57.695269+0000","last_became_active":"2024-04-26T01:54:09.003818+0000","last_became_peered":"2024-04-26T01:54:09.003818+0000","last_unstale":"2024-04-26T01:57:57.695269+0000","last_undegraded":"2024-04-26T01:57:57.695269+0000","last_fullsized":"2024-04-26T01:57:57.695269+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:40.451304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694860+0000","last_change":"2024-04-26T01:54:09.001728+0000","last_active":"2024-04-26T01:57:57.694860+0000","last_peered":"2024-04-26T01:57:57.694860+0000","last_clean":"2024-04-26T01:57:57.694860+0000","last_became_active":"2024-04-26T01:54:09.001328+0000","last_became_peered":"2024-04-26T01:54:09.001328+0000","last_unstale":"2024-04-26T01:57:57.694860+0000","last_undegraded":"2024-04-26T01:57:57.694860+0000","last_fullsized":"2024-04-26T01:57:57.694860+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:14:48.174538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694908+0000","last_change":"2024-04-26T01:54:08.997080+0000","last_active":"2024-04-26T01:57:57.694908+0000","last_peered":"2024-04-26T01:57:57.694908+0000","last_clean":"2024-04-26T01:57:57.694908+0000","last_became_active":"2024-04-26T01:54:08.996610+0000","last_became_peered":"2024-04-26T01:54:08.996610+0000","last_unstale":"2024-04-26T01:57:57.694908+0000","last_undegraded":"2024-04-26T01:57:57.694908+0000","last_fullsized":"2024-04-26T01:57:57.694908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:30.879664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950380+0000","last_change":"2024-04-26T01:54:09.003113+0000","last_active":"2024-04-26T01:57:56.950380+0000","last_peered":"2024-04-26T01:57:56.950380+0000","last_clean":"2024-04-26T01:57:56.950380+0000","last_became_active":"2024-04-26T01:54:09.002724+0000","last_became_peered":"2024-04-26T01:54:09.002724+0000","last_unstale":"2024-04-26T01:57:56.950380+0000","last_undegraded":"2024-04-26T01:57:56.950380+0000","last_fullsized":"2024-04-26T01:57:56.950380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:47.890409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768783+0000","last_change":"2024-04-26T01:54:09.012660+0000","last_active":"2024-04-26T01:57:56.768783+0000","last_peered":"2024-04-26T01:57:56.768783+0000","last_clean":"2024-04-26T01:57:56.768783+0000","last_became_active":"2024-04-26T01:54:09.012391+0000","last_became_peered":"2024-04-26T01:54:09.012391+0000","last_unstale":"2024-04-26T01:57:56.768783+0000","last_undegraded":"2024-04-26T01:57:56.768783+0000","last_fullsized":"2024-04-26T01:57:56.768783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:33:21.693415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619118+0000","last_change":"2024-04-26T01:54:09.006290+0000","last_active":"2024-04-26T01:57:56.619118+0000","last_peered":"2024-04-26T01:57:56.619118+0000","last_clean":"2024-04-26T01:57:56.619118+0000","last_became_active":"2024-04-26T01:54:09.005929+0000","last_became_peered":"2024-04-26T01:54:09.005929+0000","last_unstale":"2024-04-26T01:57:56.619118+0000","last_undegraded":"2024-04-26T01:57:56.619118+0000","last_fullsized":"2024-04-26T01:57:56.619118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:42.785258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950205+0000","last_change":"2024-04-26T01:54:09.021249+0000","last_active":"2024-04-26T01:57:56.950205+0000","last_peered":"2024-04-26T01:57:56.950205+0000","last_clean":"2024-04-26T01:57:56.950205+0000","last_became_active":"2024-04-26T01:54:09.020792+0000","last_became_peered":"2024-04-26T01:54:09.020792+0000","last_unstale":"2024-04-26T01:57:56.950205+0000","last_undegraded":"2024-04-26T01:57:56.950205+0000","last_fullsized":"2024-04-26T01:57:56.950205+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:08:44.364840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'4","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619690+0000","last_change":"2024-04-26T01:54:03.952250+0000","last_active":"2024-04-26T01:57:56.619690+0000","last_peered":"2024-04-26T01:57:56.619690+0000","last_clean":"2024-04-26T01:57:56.619690+0000","last_became_active":"2024-04-26T01:54:03.949250+0000","last_became_peered":"2024-04-26T01:54:03.949250+0000","last_unstale":"2024-04-26T01:57:56.619690+0000","last_undegraded":"2024-04-26T01:57:56.619690+0000","last_fullsized":"2024-04-26T01:57:56.619690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:18:36.149080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769502+0000","last_change":"2024-04-26T01:54:09.019019+0000","last_active":"2024-04-26T01:57:56.769502+0000","last_peered":"2024-04-26T01:57:56.769502+0000","last_clean":"2024-04-26T01:57:56.769502+0000","last_became_active":"2024-04-26T01:54:09.018081+0000","last_became_peered":"2024-04-26T01:54:09.018081+0000","last_unstale":"2024-04-26T01:57:56.769502+0000","last_undegraded":"2024-04-26T01:57:56.769502+0000","last_fullsized":"2024-04-26T01:57:56.769502+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:02:45.519117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770101+0000","last_change":"2024-04-26T01:54:04.410635+0000","last_active":"2024-04-26T01:57:56.770101+0000","last_peered":"2024-04-26T01:57:56.770101+0000","last_clean":"2024-04-26T01:57:56.770101+0000","last_became_active":"2024-04-26T01:54:04.409074+0000","last_became_peered":"2024-04-26T01:54:04.409074+0000","last_unstale":"2024-04-26T01:57:56.770101+0000","last_undegraded":"2024-04-26T01:57:56.770101+0000","last_fullsized":"2024-04-26T01:57:56.770101+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:37:35.739961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.696232+0000","last_change":"2024-04-26T01:54:09.010882+0000","last_active":"2024-04-26T01:57:57.696232+0000","last_peered":"2024-04-26T01:57:57.696232+0000","last_clean":"2024-04-26T01:57:57.696232+0000","last_became_active":"2024-04-26T01:54:09.006394+0000","last_became_peered":"2024-04-26T01:54:09.006394+0000","last_unstale":"2024-04-26T01:57:57.696232+0000","last_undegraded":"2024-04-26T01:57:57.696232+0000","last_fullsized":"2024-04-26T01:57:57.696232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:26:26.473911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622427+0000","last_change":"2024-04-26T01:54:03.946237+0000","last_active":"2024-04-26T01:57:56.622427+0000","last_peered":"2024-04-26T01:57:56.622427+0000","last_clean":"2024-04-26T01:57:56.622427+0000","last_became_active":"2024-04-26T01:54:03.945865+0000","last_became_peered":"2024-04-26T01:54:03.945865+0000","last_unstale":"2024-04-26T01:57:56.622427+0000","last_undegraded":"2024-04-26T01:57:56.622427+0000","last_fullsized":"2024-04-26T01:57:56.622427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:57:29.517641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950679+0000","last_change":"2024-04-26T01:54:09.011173+0000","last_active":"2024-04-26T01:57:56.950679+0000","last_peered":"2024-04-26T01:57:56.950679+0000","last_clean":"2024-04-26T01:57:56.950679+0000","last_became_active":"2024-04-26T01:54:09.010685+0000","last_became_peered":"2024-04-26T01:54:09.010685+0000","last_unstale":"2024-04-26T01:57:56.950679+0000","last_undegraded":"2024-04-26T01:57:56.950679+0000","last_fullsized":"2024-04-26T01:57:56.950679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:43:32.924929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622193+0000","last_change":"2024-04-26T01:54:03.945291+0000","last_active":"2024-04-26T01:57:56.622193+0000","last_peered":"2024-04-26T01:57:56.622193+0000","last_clean":"2024-04-26T01:57:56.622193+0000","last_became_active":"2024-04-26T01:54:03.944848+0000","last_became_peered":"2024-04-26T01:54:03.944848+0000","last_unstale":"2024-04-26T01:57:56.622193+0000","last_undegraded":"2024-04-26T01:57:56.622193+0000","last_fullsized":"2024-04-26T01:57:56.622193+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:55:59.789693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777417+0000","last_change":"2024-04-26T01:54:08.998830+0000","last_active":"2024-04-26T01:57:56.777417+0000","last_peered":"2024-04-26T01:57:56.777417+0000","last_clean":"2024-04-26T01:57:56.777417+0000","last_became_active":"2024-04-26T01:54:08.998486+0000","last_became_peered":"2024-04-26T01:54:08.998486+0000","last_unstale":"2024-04-26T01:57:56.777417+0000","last_undegraded":"2024-04-26T01:57:56.777417+0000","last_fullsized":"2024-04-26T01:57:56.777417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:49.378366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777047+0000","last_change":"2024-04-26T01:54:03.945253+0000","last_active":"2024-04-26T01:57:56.777047+0000","last_peered":"2024-04-26T01:57:56.777047+0000","last_clean":"2024-04-26T01:57:56.777047+0000","last_became_active":"2024-04-26T01:54:03.944740+0000","last_became_peered":"2024-04-26T01:54:03.944740+0000","last_unstale":"2024-04-26T01:57:56.777047+0000","last_undegraded":"2024-04-26T01:57:56.777047+0000","last_fullsized":"2024-04-26T01:57:56.777047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:03:04.090796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621810+0000","last_change":"2024-04-26T01:54:09.006502+0000","last_active":"2024-04-26T01:57:56.621810+0000","last_peered":"2024-04-26T01:57:56.621810+0000","last_clean":"2024-04-26T01:57:56.621810+0000","last_became_active":"2024-04-26T01:54:09.006185+0000","last_became_peered":"2024-04-26T01:54:09.006185+0000","last_unstale":"2024-04-26T01:57:56.621810+0000","last_undegraded":"2024-04-26T01:57:56.621810+0000","last_fullsized":"2024-04-26T01:57:56.621810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:33:43.341623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'1","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620119+0000","last_change":"2024-04-26T01:54:03.945543+0000","last_active":"2024-04-26T01:57:56.620119+0000","last_peered":"2024-04-26T01:57:56.620119+0000","last_clean":"2024-04-26T01:57:56.620119+0000","last_became_active":"2024-04-26T01:54:03.945067+0000","last_became_peered":"2024-04-26T01:54:03.945067+0000","last_unstale":"2024-04-26T01:57:56.620119+0000","last_undegraded":"2024-04-26T01:57:56.620119+0000","last_fullsized":"2024-04-26T01:57:56.620119+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:29.160995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777189+0000","last_change":"2024-04-26T01:54:09.021012+0000","last_active":"2024-04-26T01:57:56.777189+0000","last_peered":"2024-04-26T01:57:56.777189+0000","last_clean":"2024-04-26T01:57:56.777189+0000","last_became_active":"2024-04-26T01:54:09.020591+0000","last_became_peered":"2024-04-26T01:54:09.020591+0000","last_unstale":"2024-04-26T01:57:56.777189+0000","last_undegraded":"2024-04-26T01:57:56.777189+0000","last_fullsized":"2024-04-26T01:57:56.777189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:06.423789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694756+0000","last_change":"2024-04-26T01:54:03.959548+0000","last_active":"2024-04-26T01:57:57.694756+0000","last_peered":"2024-04-26T01:57:57.694756+0000","last_clean":"2024-04-26T01:57:57.694756+0000","last_became_active":"2024-04-26T01:54:03.959034+0000","last_became_peered":"2024-04-26T01:54:03.959034+0000","last_unstale":"2024-04-26T01:57:57.694756+0000","last_undegraded":"2024-04-26T01:57:57.694756+0000","last_fullsized":"2024-04-26T01:57:57.694756+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:45.822111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768376+0000","last_change":"2024-04-26T01:54:09.013135+0000","last_active":"2024-04-26T01:57:56.768376+0000","last_peered":"2024-04-26T01:57:56.768376+0000","last_clean":"2024-04-26T01:57:56.768376+0000","last_became_active":"2024-04-26T01:54:09.010450+0000","last_became_peered":"2024-04-26T01:54:09.010450+0000","last_unstale":"2024-04-26T01:57:56.768376+0000","last_undegraded":"2024-04-26T01:57:56.768376+0000","last_fullsized":"2024-04-26T01:57:56.768376+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:30.542102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776972+0000","last_change":"2024-04-26T01:54:03.947073+0000","last_active":"2024-04-26T01:57:56.776972+0000","last_peered":"2024-04-26T01:57:56.776972+0000","last_clean":"2024-04-26T01:57:56.776972+0000","last_became_active":"2024-04-26T01:54:03.946656+0000","last_became_peered":"2024-04-26T01:54:03.946656+0000","last_unstale":"2024-04-26T01:57:56.776972+0000","last_undegraded":"2024-04-26T01:57:56.776972+0000","last_fullsized":"2024-04-26T01:57:56.776972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:32:51.136757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.618330+0000","last_change":"2024-04-26T01:54:09.023273+0000","last_active":"2024-04-26T01:57:56.618330+0000","last_peered":"2024-04-26T01:57:56.618330+0000","last_clean":"2024-04-26T01:57:56.618330+0000","last_became_active":"2024-04-26T01:54:09.022773+0000","last_became_peered":"2024-04-26T01:54:09.022773+0000","last_unstale":"2024-04-26T01:57:56.618330+0000","last_undegraded":"2024-04-26T01:57:56.618330+0000","last_fullsized":"2024-04-26T01:57:56.618330+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:02:20.809155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620569+0000","last_change":"2024-04-26T01:54:03.949291+0000","last_active":"2024-04-26T01:57:56.620569+0000","last_peered":"2024-04-26T01:57:56.620569+0000","last_clean":"2024-04-26T01:57:56.620569+0000","last_became_active":"2024-04-26T01:54:03.948859+0000","last_became_peered":"2024-04-26T01:54:03.948859+0000","last_unstale":"2024-04-26T01:57:56.620569+0000","last_undegraded":"2024-04-26T01:57:56.620569+0000","last_fullsized":"2024-04-26T01:57:56.620569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:30.849383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950805+0000","last_change":"2024-04-26T01:54:09.006486+0000","last_active":"2024-04-26T01:57:56.950805+0000","last_peered":"2024-04-26T01:57:56.950805+0000","last_clean":"2024-04-26T01:57:56.950805+0000","last_became_active":"2024-04-26T01:54:09.006093+0000","last_became_peered":"2024-04-26T01:54:09.006093+0000","last_unstale":"2024-04-26T01:57:56.950805+0000","last_undegraded":"2024-04-26T01:57:56.950805+0000","last_fullsized":"2024-04-26T01:57:56.950805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:41:20.603681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607591+0000","last_change":"2024-04-26T01:54:03.947213+0000","last_active":"2024-04-26T01:57:57.607591+0000","last_peered":"2024-04-26T01:57:57.607591+0000","last_clean":"2024-04-26T01:57:57.607591+0000","last_became_active":"2024-04-26T01:54:03.946815+0000","last_became_peered":"2024-04-26T01:54:03.946815+0000","last_unstale":"2024-04-26T01:57:57.607591+0000","last_undegraded":"2024-04-26T01:57:57.607591+0000","last_fullsized":"2024-04-26T01:57:57.607591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:26:35.478736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.696427+0000","last_change":"2024-04-26T01:54:09.004778+0000","last_active":"2024-04-26T01:57:57.696427+0000","last_peered":"2024-04-26T01:57:57.696427+0000","last_clean":"2024-04-26T01:57:57.696427+0000","last_became_active":"2024-04-26T01:54:09.004375+0000","last_became_peered":"2024-04-26T01:54:09.004375+0000","last_unstale":"2024-04-26T01:57:57.696427+0000","last_undegraded":"2024-04-26T01:57:57.696427+0000","last_fullsized":"2024-04-26T01:57:57.696427+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:55:30.618869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'3","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777417+0000","last_change":"2024-04-26T01:54:04.414076+0000","last_active":"2024-04-26T01:57:56.777417+0000","last_peered":"2024-04-26T01:57:56.777417+0000","last_clean":"2024-04-26T01:57:56.777417+0000","last_became_active":"2024-04-26T01:54:04.413706+0000","last_became_peered":"2024-04-26T01:54:04.413706+0000","last_unstale":"2024-04-26T01:57:56.777417+0000","last_undegraded":"2024-04-26T01:57:56.777417+0000","last_fullsized":"2024-04-26T01:57:56.777417+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:43:11.279711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950907+0000","last_change":"2024-04-26T01:54:09.003407+0000","last_active":"2024-04-26T01:57:56.950907+0000","last_peered":"2024-04-26T01:57:56.950907+0000","last_clean":"2024-04-26T01:57:56.950907+0000","last_became_active":"2024-04-26T01:54:09.002896+0000","last_became_peered":"2024-04-26T01:54:09.002896+0000","last_unstale":"2024-04-26T01:57:56.950907+0000","last_undegraded":"2024-04-26T01:57:56.950907+0000","last_fullsized":"2024-04-26T01:57:56.950907+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:14:46.822369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"92'41","reported_seq":522,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:58:06.859995+0000","last_change":"2024-04-26T01:54:03.945531+0000","last_active":"2024-04-26T01:58:06.859995+0000","last_peered":"2024-04-26T01:58:06.859995+0000","last_clean":"2024-04-26T01:58:06.859995+0000","last_became_active":"2024-04-26T01:54:03.945079+0000","last_became_peered":"2024-04-26T01:54:03.945079+0000","last_unstale":"2024-04-26T01:58:06.859995+0000","last_undegraded":"2024-04-26T01:58:06.859995+0000","last_fullsized":"2024-04-26T01:58:06.859995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:10.793520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70631,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":452,"num_read_kb":296,"num_write":41,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622187+0000","last_change":"2024-04-26T01:54:08.995740+0000","last_active":"2024-04-26T01:57:56.622187+0000","last_peered":"2024-04-26T01:57:56.622187+0000","last_clean":"2024-04-26T01:57:56.622187+0000","last_became_active":"2024-04-26T01:54:08.995348+0000","last_became_peered":"2024-04-26T01:54:08.995348+0000","last_unstale":"2024-04-26T01:57:56.622187+0000","last_undegraded":"2024-04-26T01:57:56.622187+0000","last_fullsized":"2024-04-26T01:57:56.622187+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:34.086231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950167+0000","last_change":"2024-04-26T01:54:03.954193+0000","last_active":"2024-04-26T01:57:56.950167+0000","last_peered":"2024-04-26T01:57:56.950167+0000","last_clean":"2024-04-26T01:57:56.950167+0000","last_became_active":"2024-04-26T01:54:03.953837+0000","last_became_peered":"2024-04-26T01:54:03.953837+0000","last_unstale":"2024-04-26T01:57:56.950167+0000","last_undegraded":"2024-04-26T01:57:56.950167+0000","last_fullsized":"2024-04-26T01:57:56.950167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:02:03.818075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694620+0000","last_change":"2024-04-26T01:54:09.020622+0000","last_active":"2024-04-26T01:57:57.694620+0000","last_peered":"2024-04-26T01:57:57.694620+0000","last_clean":"2024-04-26T01:57:57.694620+0000","last_became_active":"2024-04-26T01:54:09.019907+0000","last_became_peered":"2024-04-26T01:54:09.019907+0000","last_unstale":"2024-04-26T01:57:57.694620+0000","last_undegraded":"2024-04-26T01:57:57.694620+0000","last_fullsized":"2024-04-26T01:57:57.694620+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:19:46.464185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777048+0000","last_change":"2024-04-26T01:54:03.955310+0000","last_active":"2024-04-26T01:57:56.777048+0000","last_peered":"2024-04-26T01:57:56.777048+0000","last_clean":"2024-04-26T01:57:56.777048+0000","last_became_active":"2024-04-26T01:54:03.954838+0000","last_became_peered":"2024-04-26T01:54:03.954838+0000","last_unstale":"2024-04-26T01:57:56.777048+0000","last_undegraded":"2024-04-26T01:57:56.777048+0000","last_fullsized":"2024-04-26T01:57:56.777048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:01:03.825446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607531+0000","last_change":"2024-04-26T01:54:09.008865+0000","last_active":"2024-04-26T01:57:57.607531+0000","last_peered":"2024-04-26T01:57:57.607531+0000","last_clean":"2024-04-26T01:57:57.607531+0000","last_became_active":"2024-04-26T01:54:09.008509+0000","last_became_peered":"2024-04-26T01:54:09.008509+0000","last_unstale":"2024-04-26T01:57:57.607531+0000","last_undegraded":"2024-04-26T01:57:57.607531+0000","last_fullsized":"2024-04-26T01:57:57.607531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:25:43.929011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025527+0000","last_change":"2024-04-26T01:54:03.947301+0000","last_active":"2024-04-26T01:57:57.025527+0000","last_peered":"2024-04-26T01:57:57.025527+0000","last_clean":"2024-04-26T01:57:57.025527+0000","last_became_active":"2024-04-26T01:54:03.946858+0000","last_became_peered":"2024-04-26T01:54:03.946858+0000","last_unstale":"2024-04-26T01:57:57.025527+0000","last_undegraded":"2024-04-26T01:57:57.025527+0000","last_fullsized":"2024-04-26T01:57:57.025527+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:03.331999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607131+0000","last_change":"2024-04-26T01:54:08.999591+0000","last_active":"2024-04-26T01:57:57.607131+0000","last_peered":"2024-04-26T01:57:57.607131+0000","last_clean":"2024-04-26T01:57:57.607131+0000","last_became_active":"2024-04-26T01:54:08.999197+0000","last_became_peered":"2024-04-26T01:54:08.999197+0000","last_unstale":"2024-04-26T01:57:57.607131+0000","last_undegraded":"2024-04-26T01:57:57.607131+0000","last_fullsized":"2024-04-26T01:57:57.607131+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:40:35.706568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'8","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621717+0000","last_change":"2024-04-26T01:54:44.197929+0000","last_active":"2024-04-26T01:57:56.621717+0000","last_peered":"2024-04-26T01:57:56.621717+0000","last_clean":"2024-04-26T01:57:56.621717+0000","last_became_active":"2024-04-26T01:54:44.189719+0000","last_became_peered":"2024-04-26T01:54:44.189719+0000","last_unstale":"2024-04-26T01:57:56.621717+0000","last_undegraded":"2024-04-26T01:57:56.621717+0000","last_fullsized":"2024-04-26T01:57:56.621717+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:23:04.216156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776974+0000","last_change":"2024-04-26T01:54:09.001347+0000","last_active":"2024-04-26T01:57:56.776974+0000","last_peered":"2024-04-26T01:57:56.776974+0000","last_clean":"2024-04-26T01:57:56.776974+0000","last_became_active":"2024-04-26T01:54:08.998716+0000","last_became_peered":"2024-04-26T01:54:08.998716+0000","last_unstale":"2024-04-26T01:57:56.776974+0000","last_undegraded":"2024-04-26T01:57:56.776974+0000","last_fullsized":"2024-04-26T01:57:56.776974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:11:12.362755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'1","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622339+0000","last_change":"2024-04-26T01:54:03.943620+0000","last_active":"2024-04-26T01:57:56.622339+0000","last_peered":"2024-04-26T01:57:56.622339+0000","last_clean":"2024-04-26T01:57:56.622339+0000","last_became_active":"2024-04-26T01:54:03.943235+0000","last_became_peered":"2024-04-26T01:54:03.943235+0000","last_unstale":"2024-04-26T01:57:56.622339+0000","last_undegraded":"2024-04-26T01:57:56.622339+0000","last_fullsized":"2024-04-26T01:57:56.622339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:11:12.548077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768569+0000","last_change":"2024-04-26T01:54:09.000719+0000","last_active":"2024-04-26T01:57:56.768569+0000","last_peered":"2024-04-26T01:57:56.768569+0000","last_clean":"2024-04-26T01:57:56.768569+0000","last_became_active":"2024-04-26T01:54:09.000350+0000","last_became_peered":"2024-04-26T01:54:09.000350+0000","last_unstale":"2024-04-26T01:57:56.768569+0000","last_undegraded":"2024-04-26T01:57:56.768569+0000","last_fullsized":"2024-04-26T01:57:56.768569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:04.903193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.619036+0000","last_change":"2024-04-26T01:54:04.411191+0000","last_active":"2024-04-26T01:57:56.619036+0000","last_peered":"2024-04-26T01:57:56.619036+0000","last_clean":"2024-04-26T01:57:56.619036+0000","last_became_active":"2024-04-26T01:54:04.410321+0000","last_became_peered":"2024-04-26T01:54:04.410321+0000","last_unstale":"2024-04-26T01:57:56.619036+0000","last_undegraded":"2024-04-26T01:57:56.619036+0000","last_fullsized":"2024-04-26T01:57:56.619036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:45:15.528648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620424+0000","last_change":"2024-04-26T01:54:09.020034+0000","last_active":"2024-04-26T01:57:56.620424+0000","last_peered":"2024-04-26T01:57:56.620424+0000","last_clean":"2024-04-26T01:57:56.620424+0000","last_became_active":"2024-04-26T01:54:09.019591+0000","last_became_peered":"2024-04-26T01:54:09.019591+0000","last_unstale":"2024-04-26T01:57:56.620424+0000","last_undegraded":"2024-04-26T01:57:56.620424+0000","last_fullsized":"2024-04-26T01:57:56.620424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:08:37.852586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607359+0000","last_change":"2024-04-26T01:54:03.950346+0000","last_active":"2024-04-26T01:57:57.607359+0000","last_peered":"2024-04-26T01:57:57.607359+0000","last_clean":"2024-04-26T01:57:57.607359+0000","last_became_active":"2024-04-26T01:54:03.949957+0000","last_became_peered":"2024-04-26T01:54:03.949957+0000","last_unstale":"2024-04-26T01:57:57.607359+0000","last_undegraded":"2024-04-26T01:57:57.607359+0000","last_fullsized":"2024-04-26T01:57:57.607359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:16.259553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.620119+0000","last_change":"2024-04-26T01:54:09.011670+0000","last_active":"2024-04-26T01:57:56.620119+0000","last_peered":"2024-04-26T01:57:56.620119+0000","last_clean":"2024-04-26T01:57:56.620119+0000","last_became_active":"2024-04-26T01:54:09.011110+0000","last_became_peered":"2024-04-26T01:54:09.011110+0000","last_unstale":"2024-04-26T01:57:56.620119+0000","last_undegraded":"2024-04-26T01:57:56.620119+0000","last_fullsized":"2024-04-26T01:57:56.620119+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:40.036551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.777211+0000","last_change":"2024-04-26T01:54:09.001825+0000","last_active":"2024-04-26T01:57:56.777211+0000","last_peered":"2024-04-26T01:57:56.777211+0000","last_clean":"2024-04-26T01:57:56.777211+0000","last_became_active":"2024-04-26T01:54:09.001450+0000","last_became_peered":"2024-04-26T01:54:09.001450+0000","last_unstale":"2024-04-26T01:57:56.777211+0000","last_undegraded":"2024-04-26T01:57:56.777211+0000","last_fullsized":"2024-04-26T01:57:56.777211+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:33:50.086968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694641+0000","last_change":"2024-04-26T01:54:04.413990+0000","last_active":"2024-04-26T01:57:57.694641+0000","last_peered":"2024-04-26T01:57:57.694641+0000","last_clean":"2024-04-26T01:57:57.694641+0000","last_became_active":"2024-04-26T01:54:04.413359+0000","last_became_peered":"2024-04-26T01:54:04.413359+0000","last_unstale":"2024-04-26T01:57:57.694641+0000","last_undegraded":"2024-04-26T01:57:57.694641+0000","last_fullsized":"2024-04-26T01:57:57.694641+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:36:11.529238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769430+0000","last_change":"2024-04-26T01:54:09.015663+0000","last_active":"2024-04-26T01:57:56.769430+0000","last_peered":"2024-04-26T01:57:56.769430+0000","last_clean":"2024-04-26T01:57:56.769430+0000","last_became_active":"2024-04-26T01:54:09.014746+0000","last_became_peered":"2024-04-26T01:54:09.014746+0000","last_unstale":"2024-04-26T01:57:56.769430+0000","last_undegraded":"2024-04-26T01:57:56.769430+0000","last_fullsized":"2024-04-26T01:57:56.769430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:53:46.483676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025760+0000","last_change":"2024-04-26T01:54:03.954385+0000","last_active":"2024-04-26T01:57:57.025760+0000","last_peered":"2024-04-26T01:57:57.025760+0000","last_clean":"2024-04-26T01:57:57.025760+0000","last_became_active":"2024-04-26T01:54:03.953972+0000","last_became_peered":"2024-04-26T01:54:03.953972+0000","last_unstale":"2024-04-26T01:57:57.025760+0000","last_undegraded":"2024-04-26T01:57:57.025760+0000","last_fullsized":"2024-04-26T01:57:57.025760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:29.430244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"92'148","reported_seq":405,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:58.824770+0000","last_change":"2024-04-26T01:52:25.488561+0000","last_active":"2024-04-26T01:57:58.824770+0000","last_peered":"2024-04-26T01:57:58.824770+0000","last_clean":"2024-04-26T01:57:58.824770+0000","last_became_active":"2024-04-26T01:52:25.488070+0000","last_became_peered":"2024-04-26T01:52:25.488070+0000","last_unstale":"2024-04-26T01:57:58.824770+0000","last_undegraded":"2024-04-26T01:57:58.824770+0000","last_fullsized":"2024-04-26T01:57:58.824770+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:48:33.062956+0000","last_clean_scrub_stamp":"2024-04-26T01:48:33.062956+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:41:43.837934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.694632+0000","last_change":"2024-04-26T01:54:09.009469+0000","last_active":"2024-04-26T01:57:57.694632+0000","last_peered":"2024-04-26T01:57:57.694632+0000","last_clean":"2024-04-26T01:57:57.694632+0000","last_became_active":"2024-04-26T01:54:09.009090+0000","last_became_peered":"2024-04-26T01:54:09.009090+0000","last_unstale":"2024-04-26T01:57:57.694632+0000","last_undegraded":"2024-04-26T01:57:57.694632+0000","last_fullsized":"2024-04-26T01:57:57.694632+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:30:05.264019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607402+0000","last_change":"2024-04-26T01:54:03.949426+0000","last_active":"2024-04-26T01:57:57.607402+0000","last_peered":"2024-04-26T01:57:57.607402+0000","last_clean":"2024-04-26T01:57:57.607402+0000","last_became_active":"2024-04-26T01:54:03.948971+0000","last_became_peered":"2024-04-26T01:54:03.948971+0000","last_unstale":"2024-04-26T01:57:57.607402+0000","last_undegraded":"2024-04-26T01:57:57.607402+0000","last_fullsized":"2024-04-26T01:57:57.607402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T13:34:32.699297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.622296+0000","last_change":"2024-04-26T01:54:04.409211+0000","last_active":"2024-04-26T01:57:56.622296+0000","last_peered":"2024-04-26T01:57:56.622296+0000","last_clean":"2024-04-26T01:57:56.622296+0000","last_became_active":"2024-04-26T01:54:04.408807+0000","last_became_peered":"2024-04-26T01:54:04.408807+0000","last_unstale":"2024-04-26T01:57:56.622296+0000","last_undegraded":"2024-04-26T01:57:56.622296+0000","last_fullsized":"2024-04-26T01:57:56.622296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:04:12.270992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695365+0000","last_change":"2024-04-26T01:54:09.011453+0000","last_active":"2024-04-26T01:57:57.695365+0000","last_peered":"2024-04-26T01:57:57.695365+0000","last_clean":"2024-04-26T01:57:57.695365+0000","last_became_active":"2024-04-26T01:54:09.011006+0000","last_became_peered":"2024-04-26T01:54:09.011006+0000","last_unstale":"2024-04-26T01:57:57.695365+0000","last_undegraded":"2024-04-26T01:57:57.695365+0000","last_fullsized":"2024-04-26T01:57:57.695365+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:27.152033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'123","reported_seq":287,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:58:00.401269+0000","last_change":"2024-04-26T01:54:44.196441+0000","last_active":"2024-04-26T01:58:00.401269+0000","last_peered":"2024-04-26T01:58:00.401269+0000","last_clean":"2024-04-26T01:58:00.401269+0000","last_became_active":"2024-04-26T01:54:44.191671+0000","last_became_peered":"2024-04-26T01:54:44.191671+0000","last_unstale":"2024-04-26T01:58:00.401269+0000","last_undegraded":"2024-04-26T01:58:00.401269+0000","last_fullsized":"2024-04-26T01:58:00.401269+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:38:38.191081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":228,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770213+0000","last_change":"2024-04-26T01:54:09.018502+0000","last_active":"2024-04-26T01:57:56.770213+0000","last_peered":"2024-04-26T01:57:56.770213+0000","last_clean":"2024-04-26T01:57:56.770213+0000","last_became_active":"2024-04-26T01:54:09.017929+0000","last_became_peered":"2024-04-26T01:54:09.017929+0000","last_unstale":"2024-04-26T01:57:56.770213+0000","last_undegraded":"2024-04-26T01:57:56.770213+0000","last_fullsized":"2024-04-26T01:57:56.770213+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:26:43.359823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.025788+0000","last_change":"2024-04-26T01:54:04.546550+0000","last_active":"2024-04-26T01:57:57.025788+0000","last_peered":"2024-04-26T01:57:57.025788+0000","last_clean":"2024-04-26T01:57:57.025788+0000","last_became_active":"2024-04-26T01:54:04.546133+0000","last_became_peered":"2024-04-26T01:54:04.546133+0000","last_unstale":"2024-04-26T01:57:57.025788+0000","last_undegraded":"2024-04-26T01:57:57.025788+0000","last_fullsized":"2024-04-26T01:57:57.025788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:05:33.524500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695999+0000","last_change":"2024-04-26T01:54:44.379735+0000","last_active":"2024-04-26T01:57:57.695999+0000","last_peered":"2024-04-26T01:57:57.695999+0000","last_clean":"2024-04-26T01:57:57.695999+0000","last_became_active":"2024-04-26T01:54:44.378486+0000","last_became_peered":"2024-04-26T01:54:44.378486+0000","last_unstale":"2024-04-26T01:57:57.695999+0000","last_undegraded":"2024-04-26T01:57:57.695999+0000","last_fullsized":"2024-04-26T01:57:57.695999+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:40:35.187830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950387+0000","last_change":"2024-04-26T01:54:44.189980+0000","last_active":"2024-04-26T01:57:56.950387+0000","last_peered":"2024-04-26T01:57:56.950387+0000","last_clean":"2024-04-26T01:57:56.950387+0000","last_became_active":"2024-04-26T01:54:44.189620+0000","last_became_peered":"2024-04-26T01:54:44.189620+0000","last_unstale":"2024-04-26T01:57:56.950387+0000","last_undegraded":"2024-04-26T01:57:56.950387+0000","last_fullsized":"2024-04-26T01:57:56.950387+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:36.948327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621727+0000","last_change":"2024-04-26T01:54:08.998770+0000","last_active":"2024-04-26T01:57:56.621727+0000","last_peered":"2024-04-26T01:57:56.621727+0000","last_clean":"2024-04-26T01:57:56.621727+0000","last_became_active":"2024-04-26T01:54:08.998407+0000","last_became_peered":"2024-04-26T01:54:08.998407+0000","last_unstale":"2024-04-26T01:57:56.621727+0000","last_undegraded":"2024-04-26T01:57:56.621727+0000","last_fullsized":"2024-04-26T01:57:56.621727+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:04:50.497796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769205+0000","last_change":"2024-04-26T01:54:03.957935+0000","last_active":"2024-04-26T01:57:56.769205+0000","last_peered":"2024-04-26T01:57:56.769205+0000","last_clean":"2024-04-26T01:57:56.769205+0000","last_became_active":"2024-04-26T01:54:03.957208+0000","last_became_peered":"2024-04-26T01:54:03.957208+0000","last_unstale":"2024-04-26T01:57:56.769205+0000","last_undegraded":"2024-04-26T01:57:56.769205+0000","last_fullsized":"2024-04-26T01:57:56.769205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:18:34.121651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768498+0000","last_change":"2024-04-26T01:54:09.005193+0000","last_active":"2024-04-26T01:57:56.768498+0000","last_peered":"2024-04-26T01:57:56.768498+0000","last_clean":"2024-04-26T01:57:56.768498+0000","last_became_active":"2024-04-26T01:54:09.004860+0000","last_became_peered":"2024-04-26T01:54:09.004860+0000","last_unstale":"2024-04-26T01:57:56.768498+0000","last_undegraded":"2024-04-26T01:57:56.768498+0000","last_fullsized":"2024-04-26T01:57:56.768498+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:21:18.192280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950186+0000","last_change":"2024-04-26T01:54:03.955244+0000","last_active":"2024-04-26T01:57:56.950186+0000","last_peered":"2024-04-26T01:57:56.950186+0000","last_clean":"2024-04-26T01:57:56.950186+0000","last_became_active":"2024-04-26T01:54:03.954886+0000","last_became_peered":"2024-04-26T01:54:03.954886+0000","last_unstale":"2024-04-26T01:57:56.950186+0000","last_undegraded":"2024-04-26T01:57:56.950186+0000","last_fullsized":"2024-04-26T01:57:56.950186+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:14:49.560986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":22,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.618702+0000","last_change":"2024-04-26T01:54:44.381170+0000","last_active":"2024-04-26T01:57:56.618702+0000","last_peered":"2024-04-26T01:57:56.618702+0000","last_clean":"2024-04-26T01:57:56.618702+0000","last_became_active":"2024-04-26T01:54:44.379655+0000","last_became_peered":"2024-04-26T01:54:44.379655+0000","last_unstale":"2024-04-26T01:57:56.618702+0000","last_undegraded":"2024-04-26T01:57:56.618702+0000","last_fullsized":"2024-04-26T01:57:56.618702+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:43:19.785383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,10],"acting":[9,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.b","version":"89'2","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.621921+0000","last_change":"2024-04-26T01:54:03.946199+0000","last_active":"2024-04-26T01:57:56.621921+0000","last_peered":"2024-04-26T01:57:56.621921+0000","last_clean":"2024-04-26T01:57:56.621921+0000","last_became_active":"2024-04-26T01:54:03.945770+0000","last_became_peered":"2024-04-26T01:54:03.945770+0000","last_unstale":"2024-04-26T01:57:56.621921+0000","last_undegraded":"2024-04-26T01:57:56.621921+0000","last_fullsized":"2024-04-26T01:57:56.621921+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:29:41.614968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607367+0000","last_change":"2024-04-26T01:54:09.008828+0000","last_active":"2024-04-26T01:57:57.607367+0000","last_peered":"2024-04-26T01:57:57.607367+0000","last_clean":"2024-04-26T01:57:57.607367+0000","last_became_active":"2024-04-26T01:54:09.008450+0000","last_became_peered":"2024-04-26T01:54:09.008450+0000","last_unstale":"2024-04-26T01:57:57.607367+0000","last_undegraded":"2024-04-26T01:57:57.607367+0000","last_fullsized":"2024-04-26T01:57:57.607367+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:16:28.396767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950192+0000","last_change":"2024-04-26T01:54:04.546404+0000","last_active":"2024-04-26T01:57:56.950192+0000","last_peered":"2024-04-26T01:57:56.950192+0000","last_clean":"2024-04-26T01:57:56.950192+0000","last_became_active":"2024-04-26T01:54:04.545981+0000","last_became_peered":"2024-04-26T01:54:04.545981+0000","last_unstale":"2024-04-26T01:57:56.950192+0000","last_undegraded":"2024-04-26T01:57:56.950192+0000","last_fullsized":"2024-04-26T01:57:56.950192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:44:04.446845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607705+0000","last_change":"2024-04-26T01:54:08.999835+0000","last_active":"2024-04-26T01:57:57.607705+0000","last_peered":"2024-04-26T01:57:57.607705+0000","last_clean":"2024-04-26T01:57:57.607705+0000","last_became_active":"2024-04-26T01:54:08.997874+0000","last_became_peered":"2024-04-26T01:54:08.997874+0000","last_unstale":"2024-04-26T01:57:57.607705+0000","last_undegraded":"2024-04-26T01:57:57.607705+0000","last_fullsized":"2024-04-26T01:57:57.607705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:03.278919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"92'73","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:58:00.395163+0000","last_change":"2024-04-26T01:54:03.955258+0000","last_active":"2024-04-26T01:58:00.395163+0000","last_peered":"2024-04-26T01:58:00.395163+0000","last_clean":"2024-04-26T01:58:00.395163+0000","last_became_active":"2024-04-26T01:54:03.954811+0000","last_became_peered":"2024-04-26T01:54:03.954811+0000","last_unstale":"2024-04-26T01:58:00.395163+0000","last_undegraded":"2024-04-26T01:58:00.395163+0000","last_fullsized":"2024-04-26T01:58:00.395163+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:23.225049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607490+0000","last_change":"2024-04-26T01:54:09.019379+0000","last_active":"2024-04-26T01:57:57.607490+0000","last_peered":"2024-04-26T01:57:57.607490+0000","last_clean":"2024-04-26T01:57:57.607490+0000","last_became_active":"2024-04-26T01:54:09.018977+0000","last_became_peered":"2024-04-26T01:54:09.018977+0000","last_unstale":"2024-04-26T01:57:57.607490+0000","last_undegraded":"2024-04-26T01:57:57.607490+0000","last_fullsized":"2024-04-26T01:57:57.607490+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:34.019726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695129+0000","last_change":"2024-04-26T01:54:03.956301+0000","last_active":"2024-04-26T01:57:57.695129+0000","last_peered":"2024-04-26T01:57:57.695129+0000","last_clean":"2024-04-26T01:57:57.695129+0000","last_became_active":"2024-04-26T01:54:03.955535+0000","last_became_peered":"2024-04-26T01:54:03.955535+0000","last_unstale":"2024-04-26T01:57:57.695129+0000","last_undegraded":"2024-04-26T01:57:57.695129+0000","last_fullsized":"2024-04-26T01:57:57.695129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:02.908139+0000","last_clean_scrub_stamp":"2024-04-26T01:54:02.908139+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:08.377427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.607764+0000","last_change":"2024-04-26T01:54:08.996530+0000","last_active":"2024-04-26T01:57:57.607764+0000","last_peered":"2024-04-26T01:57:57.607764+0000","last_clean":"2024-04-26T01:57:57.607764+0000","last_became_active":"2024-04-26T01:54:08.995973+0000","last_became_peered":"2024-04-26T01:54:08.995973+0000","last_unstale":"2024-04-26T01:57:57.607764+0000","last_undegraded":"2024-04-26T01:57:57.607764+0000","last_fullsized":"2024-04-26T01:57:57.607764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:10:41.078522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768557+0000","last_change":"2024-04-26T01:54:44.189043+0000","last_active":"2024-04-26T01:57:56.768557+0000","last_peered":"2024-04-26T01:57:56.768557+0000","last_clean":"2024-04-26T01:57:56.768557+0000","last_became_active":"2024-04-26T01:54:44.188563+0000","last_became_peered":"2024-04-26T01:54:44.188563+0000","last_unstale":"2024-04-26T01:57:56.768557+0000","last_undegraded":"2024-04-26T01:57:56.768557+0000","last_fullsized":"2024-04-26T01:57:56.768557+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:14:08.321279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,9],"acting":[11,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":35,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.949978+0000","last_change":"2024-04-26T01:54:09.005179+0000","last_active":"2024-04-26T01:57:56.949978+0000","last_peered":"2024-04-26T01:57:56.949978+0000","last_clean":"2024-04-26T01:57:56.949978+0000","last_became_active":"2024-04-26T01:54:09.003038+0000","last_became_peered":"2024-04-26T01:54:09.003038+0000","last_unstale":"2024-04-26T01:57:56.949978+0000","last_undegraded":"2024-04-26T01:57:56.949978+0000","last_fullsized":"2024-04-26T01:57:56.949978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:35:56.521749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950009+0000","last_change":"2024-04-26T01:54:09.019413+0000","last_active":"2024-04-26T01:57:56.950009+0000","last_peered":"2024-04-26T01:57:56.950009+0000","last_clean":"2024-04-26T01:57:56.950009+0000","last_became_active":"2024-04-26T01:54:09.019052+0000","last_became_peered":"2024-04-26T01:54:09.019052+0000","last_unstale":"2024-04-26T01:57:56.950009+0000","last_undegraded":"2024-04-26T01:57:56.950009+0000","last_fullsized":"2024-04-26T01:57:56.950009+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:32.615020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695347+0000","last_change":"2024-04-26T01:54:09.009926+0000","last_active":"2024-04-26T01:57:57.695347+0000","last_peered":"2024-04-26T01:57:57.695347+0000","last_clean":"2024-04-26T01:57:57.695347+0000","last_became_active":"2024-04-26T01:54:09.008811+0000","last_became_peered":"2024-04-26T01:54:09.008811+0000","last_unstale":"2024-04-26T01:57:57.695347+0000","last_undegraded":"2024-04-26T01:57:57.695347+0000","last_fullsized":"2024-04-26T01:57:57.695347+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:58:12.509989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695519+0000","last_change":"2024-04-26T01:54:09.014745+0000","last_active":"2024-04-26T01:57:57.695519+0000","last_peered":"2024-04-26T01:57:57.695519+0000","last_clean":"2024-04-26T01:57:57.695519+0000","last_became_active":"2024-04-26T01:54:09.013835+0000","last_became_peered":"2024-04-26T01:54:09.013835+0000","last_unstale":"2024-04-26T01:57:57.695519+0000","last_undegraded":"2024-04-26T01:57:57.695519+0000","last_fullsized":"2024-04-26T01:57:57.695519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:00:10.156557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768629+0000","last_change":"2024-04-26T01:54:08.999637+0000","last_active":"2024-04-26T01:57:56.768629+0000","last_peered":"2024-04-26T01:57:56.768629+0000","last_clean":"2024-04-26T01:57:56.768629+0000","last_became_active":"2024-04-26T01:54:08.999141+0000","last_became_peered":"2024-04-26T01:54:08.999141+0000","last_unstale":"2024-04-26T01:57:56.768629+0000","last_undegraded":"2024-04-26T01:57:56.768629+0000","last_fullsized":"2024-04-26T01:57:56.768629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:18:48.281534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950313+0000","last_change":"2024-04-26T01:54:09.002578+0000","last_active":"2024-04-26T01:57:56.950313+0000","last_peered":"2024-04-26T01:57:56.950313+0000","last_clean":"2024-04-26T01:57:56.950313+0000","last_became_active":"2024-04-26T01:54:09.000249+0000","last_became_peered":"2024-04-26T01:54:09.000249+0000","last_unstale":"2024-04-26T01:57:56.950313+0000","last_undegraded":"2024-04-26T01:57:56.950313+0000","last_fullsized":"2024-04-26T01:57:56.950313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:21:28.732266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.770087+0000","last_change":"2024-04-26T01:54:44.190885+0000","last_active":"2024-04-26T01:57:56.770087+0000","last_peered":"2024-04-26T01:57:56.770087+0000","last_clean":"2024-04-26T01:57:56.770087+0000","last_became_active":"2024-04-26T01:54:44.188172+0000","last_became_peered":"2024-04-26T01:54:44.188172+0000","last_unstale":"2024-04-26T01:57:56.770087+0000","last_undegraded":"2024-04-26T01:57:56.770087+0000","last_fullsized":"2024-04-26T01:57:56.770087+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:20:30.609370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,8,2],"acting":[3,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":29,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.776988+0000","last_change":"2024-04-26T01:54:09.020925+0000","last_active":"2024-04-26T01:57:56.776988+0000","last_peered":"2024-04-26T01:57:56.776988+0000","last_clean":"2024-04-26T01:57:56.776988+0000","last_became_active":"2024-04-26T01:54:09.020536+0000","last_became_peered":"2024-04-26T01:54:09.020536+0000","last_unstale":"2024-04-26T01:57:56.776988+0000","last_undegraded":"2024-04-26T01:57:56.776988+0000","last_fullsized":"2024-04-26T01:57:56.776988+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:59:31.311391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":31,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.769011+0000","last_change":"2024-04-26T01:54:09.012647+0000","last_active":"2024-04-26T01:57:56.769011+0000","last_peered":"2024-04-26T01:57:56.769011+0000","last_clean":"2024-04-26T01:57:56.769011+0000","last_became_active":"2024-04-26T01:54:09.009589+0000","last_became_peered":"2024-04-26T01:54:09.009589+0000","last_unstale":"2024-04-26T01:57:56.769011+0000","last_undegraded":"2024-04-26T01:57:56.769011+0000","last_fullsized":"2024-04-26T01:57:56.769011+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:59:31.599584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.950624+0000","last_change":"2024-04-26T01:54:09.019102+0000","last_active":"2024-04-26T01:57:56.950624+0000","last_peered":"2024-04-26T01:57:56.950624+0000","last_clean":"2024-04-26T01:57:56.950624+0000","last_became_active":"2024-04-26T01:54:09.018741+0000","last_became_peered":"2024-04-26T01:54:09.018741+0000","last_unstale":"2024-04-26T01:57:56.950624+0000","last_undegraded":"2024-04-26T01:57:56.950624+0000","last_fullsized":"2024-04-26T01:57:56.950624+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:57:40.080000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":34,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:57.695212+0000","last_change":"2024-04-26T01:54:09.014110+0000","last_active":"2024-04-26T01:57:57.695212+0000","last_peered":"2024-04-26T01:57:57.695212+0000","last_clean":"2024-04-26T01:57:57.695212+0000","last_became_active":"2024-04-26T01:54:09.004291+0000","last_became_peered":"2024-04-26T01:54:09.004291+0000","last_unstale":"2024-04-26T01:57:57.695212+0000","last_undegraded":"2024-04-26T01:57:57.695212+0000","last_fullsized":"2024-04-26T01:57:57.695212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:28:58.646136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-26T01:57:56.768462+0000","last_change":"2024-04-26T01:54:44.188650+0000","last_active":"2024-04-26T01:57:56.768462+0000","last_peered":"2024-04-26T01:57:56.768462+0000","last_clean":"2024-04-26T01:57:56.768462+0000","last_became_active":"2024-04-26T01:54:44.188210+0000","last_became_peered":"2024-04-26T01:54:44.188210+0000","last_unstale":"2024-04-26T01:57:56.768462+0000","last_undegraded":"2024-04-26T01:57:56.768462+0000","last_fullsized":"2024-04-26T01:57:56.768462+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T01:54:07.947281+0000","last_clean_scrub_stamp":"2024-04-26T01:54:07.947281+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:43:43.757129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":792877,"num_objects":24,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":924,"num_read_kb":721,"num_write":390,"num_write_kb":2589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2482176,"data_stored":2378631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":268,"ondisk_log_size":268,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351043,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36696,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708456,"statfs":{"total":95995035648,"available":95957458944,"internally_reserved":0,"allocated":1392640,"data_stored":885080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612679,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29112,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716040,"statfs":{"total":95995035648,"available":95965224960,"internally_reserved":0,"allocated":2023424,"data_stored":1481406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36700,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708452,"statfs":{"total":95995035648,"available":95957454848,"internally_reserved":0,"allocated":1396736,"data_stored":885662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037842,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":492468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1601536,"data_stored":1082931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1003520,"data_stored":491981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574969,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1081344,"data_stored":562495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738494,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":491954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":562529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149806,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37036,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708116,"statfs":{"total":95995035648,"available":95957110784,"internally_reserved":0,"allocated":1740800,"data_stored":1213650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888169,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":491886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724644,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28912,"kb_used_data":1896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716240,"statfs":{"total":95995035648,"available":95965429760,"internally_reserved":0,"allocated":1941504,"data_stored":1410763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":70665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":733184,"data_stored":721552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":344064,"data_stored":328414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":393798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":399057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T01:58:11.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:58:11.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:11 smithi037 ceph-mon[27906]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-04-26T01:58:11.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:11 smithi037 ceph-mon[27906]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:12.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:11 smithi045 ceph-mon[32520]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-04-26T01:58:12.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:11 smithi045 ceph-mon[32520]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:11 smithi071 ceph-mon[32578]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-04-26T01:58:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:11 smithi071 ceph-mon[32578]: from='client.14961 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T01:58:12.413 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.757+0000 7fc7a1000640 1 -- 172.21.15.37:0/601566679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c074c20 msgr2=0x7fc79c10e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.757+0000 7fc7a1000640 1 --2- 172.21.15.37:0/601566679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c074c20 0x7fc79c10e2b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc788009860 tx=0x7fc78802a3d0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 -- 172.21.15.37:0/601566679 shutdown_connections 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 --2- 172.21.15.37:0/601566679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc79c10e7f0 0x7fc79c110be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 --2- 172.21.15.37:0/601566679 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c074c20 0x7fc79c10e2b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 --2- 172.21.15.37:0/601566679 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074300 0x7fc79c0746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 -- 172.21.15.37:0/601566679 >> 172.21.15.37:0/601566679 conn(0x7fc79c0fd9e0 msgr2=0x7fc79c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 -- 172.21.15.37:0/601566679 shutdown_connections 2024-04-26T01:58:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.758+0000 7fc7a1000640 1 -- 172.21.15.37:0/601566679 wait complete. 2024-04-26T01:58:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 Processor -- start 2024-04-26T01:58:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 -- start start 2024-04-26T01:58:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc79c074300 0x7fc79c1a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 0x7fc79c1a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c10e7f0 0x7fc79c19a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1143e0 con 0x7fc79c074300 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c114260 con 0x7fc79c074c20 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.759+0000 7fc7a1000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c114560 con 0x7fc79c10e7f0 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 0x7fc79c1a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc79c074300 0x7fc79c1a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 0x7fc79c1a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:53864/0 (socket says 172.21.15.37:53864) 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 -- 172.21.15.37:0/3332864155 learned_addr learned my addr 172.21.15.37:0/3332864155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc79a800640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c10e7f0 0x7fc79c19a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 -- 172.21.15.37:0/3332864155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c10e7f0 msgr2=0x7fc79c19a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.760 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c10e7f0 0x7fc79c19a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 -- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc79c074300 msgr2=0x7fc79c1a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc79c074300 0x7fc79c1a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.760+0000 7fc799400640 1 -- 172.21.15.37:0/3332864155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc79c19af90 con 0x7fc79c074c20 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.761+0000 7fc799400640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 0x7fc79c1a0b90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc788030fc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.761+0000 7fc792a00640 1 -- 172.21.15.37:0/3332864155 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788040070 con 0x7fc79c074c20 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.761+0000 7fc792a00640 1 -- 172.21.15.37:0/3332864155 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7880055f0 con 0x7fc79c074c20 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.761+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc79c19b220 con 0x7fc79c074c20 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.761+0000 7fc792a00640 1 -- 172.21.15.37:0/3332864155 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7880046c0 con 0x7fc79c074c20 2024-04-26T01:58:12.761 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.761+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc79c1a74a0 con 0x7fc79c074c20 2024-04-26T01:58:12.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.764+0000 7fc792a00640 1 -- 172.21.15.37:0/3332864155 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fc788043070 con 0x7fc79c074c20 2024-04-26T01:58:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.766+0000 7fc792a00640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc7800a3eb0 0x7fc7800a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.766+0000 7fc792a00640 1 -- 172.21.15.37:0/3332864155 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc7880f15c0 con 0x7fc79c074c20 2024-04-26T01:58:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.766+0000 7fc799e00640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc7800a3eb0 0x7fc7800a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.766+0000 7fc7a3dcc740 1 Processor -- start 2024-04-26T01:58:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 -- start start 2024-04-26T01:58:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc799e00640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc7800a3eb0 0x7fc7800a6370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005fd0 tx=0x7fc78c009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e901c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e90c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fc8e867900 con 0x55fc8e902140 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc8e80def0 con 0x55fc8e90ca00 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fc8e8638c0 con 0x55fc8e901820 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc79a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc79a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:53868/0 (socket says 172.21.15.37:53868) 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc79a800640 1 -- 172.21.15.37:0/2465646061 learned_addr learned my addr 172.21.15.37:0/2465646061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc799400640 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e90c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.767+0000 7fc799e00640 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e901c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc79a800640 1 -- 172.21.15.37:0/2465646061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 msgr2=0x55fc8e901c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc79a800640 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e901c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc79a800640 1 -- 172.21.15.37:0/2465646061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 msgr2=0x55fc8e90c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc79a800640 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e90c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc79a800640 1 -- 172.21.15.37:0/2465646061 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc8e57a210 con 0x55fc8e90ca00 2024-04-26T01:58:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc79a800640 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90edf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc794009f70 tx=0x7fc794010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bcdb8144eab1831 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc787400640 1 -- 172.21.15.37:0/2465646061 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794016070 con 0x55fc8e90ca00 2024-04-26T01:58:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc787400640 1 -- 172.21.15.37:0/2465646061 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc794004d20 con 0x55fc8e90ca00 2024-04-26T01:58:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.768+0000 7fc787400640 1 -- 172.21.15.37:0/2465646061 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7940127d0 con 0x55fc8e90ca00 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/2465646061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 msgr2=0x55fc8e90edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90edf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc794009f70 tx=0x7fc794010800 comp rx=0 tx=0).stop 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/2465646061 shutdown_connections 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90edf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e90c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/2465646061 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e901c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/2465646061 >> 172.21.15.37:0/2465646061 conn(0x55fc8e8fdd30 msgr2=0x55fc8e8fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:12.774 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/2465646061 shutdown_connections 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.769+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/2465646061 wait complete. 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.770+0000 7fc7a3dcc740 1 -- start start 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.770+0000 7fc7a3dcc740 1 Processor -- start 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e909a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e909f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fc8e90f630 con 0x55fc8e902140 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fc8e821310 con 0x55fc8e90ca00 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fc8e90f7b0 con 0x55fc8e901820 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc7a3dcc740 10 client.0 _ll_register_callbacks cb 0x55fc8e7efcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc799e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e909a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:53884/0 (socket says 172.21.15.37:53884) 2024-04-26T01:58:12.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 -- 172.21.15.37:0/3832756508 learned_addr learned my addr 172.21.15.37:0/3832756508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc799400640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e909f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 -- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 msgr2=0x55fc8e909a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e909a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 -- 172.21.15.37:0/3832756508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 msgr2=0x55fc8e909f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e909f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc79a800640 1 -- 172.21.15.37:0/3832756508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc8e646620 con 0x55fc8e90ca00 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.771+0000 7fc799e00640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e909a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc79a800640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90a490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc794012490 tx=0x7fc794004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=421be5656274489a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc77be00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc77be00640 1 -- 172.21.15.37:0/3832756508 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc794016070 con 0x55fc8e90ca00 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc794004d20 con 0x55fc8e90ca00 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc77be00640 1 -- 172.21.15.37:0/3832756508 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc7940051f0 con 0x55fc8e90ca00 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc77be00640 1 -- 172.21.15.37:0/3832756508 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc794009b70 con 0x55fc8e90ca00 2024-04-26T01:58:12.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc7a3dcc740 10 client.25064 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.772+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55fc8e924130 con 0x55fc8e90ca00 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc77be00640 1 -- 172.21.15.37:0/3832756508 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc794004d20 con 0x55fc8e90ca00 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc77be00640 1 -- 172.21.15.37:0/3832756508 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fc79400f040 con 0x55fc8e90ca00 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc7a3dcc740 10 client.25064 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc7a3dcc740 10 client.25064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc7a3dcc740 20 client.25064 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e925530 0x55fc8e927920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc7a3dcc740 4 client.25064 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc8e87b8b0 con 0x55fc8e925530 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.773+0000 7fc799e00640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e925530 0x55fc8e927920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.774+0000 7fc799e00640 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e925530 0x55fc8e927920 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0d314d720d4314 server_cookie=e6384fa1dd0a6777 in_seq=0 out_seq=0 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.774+0000 7fc77be00640 10 client.25064 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:12.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc77be00640 1 -- 172.21.15.37:0/3832756508 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fc8e87b8b0 con 0x55fc8e925530 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc77be00640 10 client.25064 handle_command_reply: tid=0 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 client.25064 shutdown 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 msgr2=0x55fc8e90a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90a490 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc794012490 tx=0x7fc794004c70 comp rx=0 tx=0).stop 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 shutdown_connections 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e925530 0x55fc8e927920 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e90ca00 0x55fc8e90a490 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc77be00640 0 client.25064 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e902140 0x55fc8e909f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/3832756508 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e909a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 >> 172.21.15.37:0/3832756508 conn(0x55fc8e8fdd30 msgr2=0x55fc8e903ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 shutdown_connections 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/3832756508 wait complete. 2024-04-26T01:58:12.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.775+0000 7fc7a3dcc740 20 client.25064 trim_cache size 0 max 16384 2024-04-26T01:58:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 Processor -- start 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 -- start start 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e9184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e9041e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e90ca00 0x55fc8e92d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1a99a0 con 0x55fc8e90ca00 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1ac220 con 0x55fc8e902140 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.798+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1aa010 con 0x55fc8e901820 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e9041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e9041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:53900/0 (socket says 172.21.15.37:53900) 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 -- 172.21.15.37:0/1662701157 learned_addr learned my addr 172.21.15.37:0/1662701157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc79a800640 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e90ca00 0x55fc8e92d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799e00640 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e9184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 -- 172.21.15.37:0/1662701157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 msgr2=0x55fc8e9184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e9184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 -- 172.21.15.37:0/1662701157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e90ca00 msgr2=0x55fc8e92d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e90ca00 0x55fc8e92d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 -- 172.21.15.37:0/1662701157 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc8e8fda20 con 0x55fc8e902140 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.799+0000 7fc799400640 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e9041e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc788005b60 tx=0x7fc7880bcdd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd9d65fddab3451c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc786a00640 1 -- 172.21.15.37:0/1662701157 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78804a040 con 0x55fc8e902140 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc786a00640 1 -- 172.21.15.37:0/1662701157 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7880d1e50 con 0x55fc8e902140 2024-04-26T01:58:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc786a00640 1 -- 172.21.15.37:0/1662701157 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7880c5b90 con 0x55fc8e902140 2024-04-26T01:58:12.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/1662701157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 msgr2=0x55fc8e9041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e9041e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc788005b60 tx=0x7fc7880bcdd0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/1662701157 shutdown_connections 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e90ca00 0x55fc8e92d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e9041e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/1662701157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e901820 0x55fc8e9184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.800+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/1662701157 >> 172.21.15.37:0/1662701157 conn(0x55fc8e8fdd30 msgr2=0x55fc8e90a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.801+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/1662701157 shutdown_connections 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.801+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/1662701157 wait complete. 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.801+0000 7fc7a3dcc740 1 -- start start 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.801+0000 7fc7a3dcc740 1 Processor -- start 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e901820 0x55fc8e9125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e912ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e90ca00 0x55fc8e913020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1aa6e0 con 0x55fc8e901820 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1abb50 con 0x55fc8e902140 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc79c1aadb0 con 0x55fc8e90ca00 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc7a3dcc740 10 client.0 _ll_register_callbacks cb 0x7fc794004ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e912ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e901820 0x55fc8e9125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e912ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:53912/0 (socket says 172.21.15.37:53912) 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 -- 172.21.15.37:0/343172767 learned_addr learned my addr 172.21.15.37:0/343172767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc79a800640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e90ca00 0x55fc8e913020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 -- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e90ca00 msgr2=0x55fc8e913020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e90ca00 0x55fc8e913020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 -- 172.21.15.37:0/343172767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e901820 msgr2=0x55fc8e9125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e901820 0x55fc8e9125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799400640 1 -- 172.21.15.37:0/343172767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc8e98aa10 con 0x55fc8e902140 2024-04-26T01:58:12.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.802+0000 7fc799e00640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e901820 0x55fc8e9125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc799400640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e912ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc788030920 tx=0x7fc788031650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3165fb587719a70d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc77be00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc77be00640 1 -- 172.21.15.37:0/343172767 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7880da020 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc77be00640 1 -- 172.21.15.37:0/343172767 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fc7880dc2c0 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7880d1e50 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc77be00640 1 -- 172.21.15.37:0/343172767 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fc788031ba0 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc7a3dcc740 10 client.25070 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55fc8e98bd00 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc77be00640 1 -- 172.21.15.37:0/343172767 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7880d1e50 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc77be00640 1 -- 172.21.15.37:0/343172767 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fc7880eee40 con 0x55fc8e902140 2024-04-26T01:58:12.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.803+0000 7fc7a3dcc740 10 client.25070 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc7a3dcc740 10 client.25070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc7a3dcc740 20 client.25070 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e98c4b0 0x55fc8e98e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc7a3dcc740 4 client.25070 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc8e875320 con 0x55fc8e98c4b0 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc799e00640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e98c4b0 0x55fc8e98e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc799e00640 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e98c4b0 0x55fc8e98e8d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d65b8ca10b44 server_cookie=b9954fac0cf0d745 in_seq=0 out_seq=0 2024-04-26T01:58:12.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.804+0000 7fc77be00640 10 client.25070 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc77be00640 1 -- 172.21.15.37:0/343172767 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc8e875320 con 0x55fc8e98c4b0 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc77be00640 10 client.25070 handle_command_reply: tid=0 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 client.25070 shutdown 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 msgr2=0x55fc8e912ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e912ae0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc788030920 tx=0x7fc788031650 comp rx=0 tx=0).stop 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 shutdown_connections 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fc8e98c4b0 0x55fc8e98e8d0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fc8e90ca00 0x55fc8e913020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fc8e902140 0x55fc8e912ae0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 --2- 172.21.15.37:0/343172767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc8e901820 0x55fc8e9125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc77be00640 0 client.25070 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 >> 172.21.15.37:0/343172767 conn(0x55fc8e8fdd30 msgr2=0x55fc8e909800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 shutdown_connections 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.815+0000 7fc7a3dcc740 1 -- 172.21.15.37:0/343172767 wait complete. 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.816+0000 7fc7a3dcc740 20 client.25070 trim_cache size 0 max 16384 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:58:12.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:58:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.816+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc7800a3eb0 msgr2=0x7fc7800a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.816+0000 7fc7a1000640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc7800a3eb0 0x7fc7800a6370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc78c005fd0 tx=0x7fc78c009260 comp rx=0 tx=0).stop 2024-04-26T01:58:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.816+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 msgr2=0x7fc79c1a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.816+0000 7fc7a1000640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 0x7fc79c1a0b90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc788000c00 tx=0x7fc788030fc0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 shutdown_connections 2024-04-26T01:58:12.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc7800a3eb0 0x7fc7800a6370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc79c10e7f0 0x7fc79c19a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc79c074c20 0x7fc79c1a0b90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 --2- 172.21.15.37:0/3332864155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc79c074300 0x7fc79c1a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 >> 172.21.15.37:0/3332864155 conn(0x7fc79c0fd9e0 msgr2=0x7fc79c0ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 shutdown_connections 2024-04-26T01:58:12.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:12.817+0000 7fc7a1000640 1 -- 172.21.15.37:0/3332864155 wait complete. 2024-04-26T01:58:13.968 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:13 smithi037 ceph-mon[27906]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-04-26T01:58:14.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:13 smithi045 ceph-mon[32520]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-04-26T01:58:14.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:13 smithi071 ceph-mon[32578]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 98 KiB/s wr, 58 op/s 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 -- 172.21.15.37:0/1215208861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094104ec0 msgr2=0x7fa0941052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 --2- 172.21.15.37:0/1215208861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094104ec0 0x7fa0941052a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fa084009920 tx=0x7fa084009c30 comp rx=0 tx=0).stop 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 -- 172.21.15.37:0/1215208861 shutdown_connections 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 --2- 172.21.15.37:0/1215208861 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa094109fa0 0x7fa094111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 --2- 172.21.15.37:0/1215208861 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094104ec0 0x7fa0941052a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 --2- 172.21.15.37:0/1215208861 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094105870 0x7fa0941098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 -- 172.21.15.37:0/1215208861 >> 172.21.15.37:0/1215208861 conn(0x7fa094100750 msgr2=0x7fa094102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 -- 172.21.15.37:0/1215208861 shutdown_connections 2024-04-26T01:58:14.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.913+0000 7fa09a400700 1 -- 172.21.15.37:0/1215208861 wait complete. 2024-04-26T01:58:14.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 Processor -- start 2024-04-26T01:58:14.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 -- start start 2024-04-26T01:58:14.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 0x7fa09419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:14.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 0x7fa09419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:14.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa094109fa0 0x7fa09419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:14.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa094114eb0 con 0x7fa094104ec0 2024-04-26T01:58:14.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa094114d30 con 0x7fa094109fa0 2024-04-26T01:58:14.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.915+0000 7fa09a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa094115030 con 0x7fa094105870 2024-04-26T01:58:14.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa093400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 0x7fa09419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 0x7fa09419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa093400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 0x7fa09419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60816/0 (socket says 172.21.15.37:60816) 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa093400700 1 -- 172.21.15.37:0/4206606104 learned_addr learned my addr 172.21.15.37:0/4206606104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 0x7fa09419b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37504/0 (socket says 172.21.15.37:37504) 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa093e00700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa094109fa0 0x7fa09419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 -- 172.21.15.37:0/4206606104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa094109fa0 msgr2=0x7fa09419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa094109fa0 0x7fa09419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 -- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 msgr2=0x7fa09419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 0x7fa09419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa092a00700 1 -- 172.21.15.37:0/4206606104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0941a0430 con 0x7fa094105870 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.916+0000 7fa093400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 0x7fa09419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:14.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.917+0000 7fa092a00700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 0x7fa09419b960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa080005fd0 tx=0x7fa08000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:14.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.917+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08000cbd0 con 0x7fa094105870 2024-04-26T01:58:14.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.917+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0800044d0 con 0x7fa094105870 2024-04-26T01:58:14.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.917+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0941a0720 con 0x7fa094105870 2024-04-26T01:58:14.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.917+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa080010430 con 0x7fa094105870 2024-04-26T01:58:14.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.917+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0941a7fa0 con 0x7fa094105870 2024-04-26T01:58:14.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.918+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0941063a0 con 0x7fa094105870 2024-04-26T01:58:14.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.920+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fa080003eb0 con 0x7fa094105870 2024-04-26T01:58:14.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.922+0000 7fa08be00700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa0780a3f50 0x7fa0780a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:14.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.922+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa0800d0c60 con 0x7fa094105870 2024-04-26T01:58:14.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.922+0000 7fa093400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa0780a3f50 0x7fa0780a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:14.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.923+0000 7fa093400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa0780a3f50 0x7fa0780a6410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa0840098f0 tx=0x7fa084019040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:14.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:14.926+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa08009d080 con 0x7fa094105870 2024-04-26T01:58:15.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.210+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa09402df80 con 0x7fa094105870 2024-04-26T01:58:15.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.211+0000 7fa08be00700 1 -- 172.21.15.37:0/4206606104 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fa08009c980 con 0x7fa094105870 2024-04-26T01:58:15.209 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:15.210 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:58:15.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa0780a3f50 msgr2=0x7fa0780a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:15.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa0780a3f50 0x7fa0780a6410 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa0840098f0 tx=0x7fa084019040 comp rx=0 tx=0).stop 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 msgr2=0x7fa09419b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 0x7fa09419b960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa080005fd0 tx=0x7fa08000ee70 comp rx=0 tx=0).stop 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 shutdown_connections 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa094104ec0 0x7fa09419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa0780a3f50 0x7fa0780a6410 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa094109fa0 0x7fa09419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 --2- 172.21.15.37:0/4206606104 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa094105870 0x7fa09419b960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 >> 172.21.15.37:0/4206606104 conn(0x7fa094100750 msgr2=0x7fa094101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:15.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 shutdown_connections 2024-04-26T01:58:15.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:15.214+0000 7fa09a400700 1 -- 172.21.15.37:0/4206606104 wait complete. 2024-04-26T01:58:15.214 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:58:15.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:15 smithi037 ceph-mon[27906]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 59 op/s 2024-04-26T01:58:15.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:15 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4206606104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:16.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:15 smithi045 ceph-mon[32520]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 59 op/s 2024-04-26T01:58:16.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:15 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4206606104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:16.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:58:16.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:15 smithi071 ceph-mon[32578]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 98 KiB/s wr, 59 op/s 2024-04-26T01:58:16.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:15 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4206606104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:16.579 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:17.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:17 smithi037 ceph-mon[27906]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T01:58:18.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:17 smithi045 ceph-mon[32520]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T01:58:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:17 smithi071 ceph-mon[32578]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T01:58:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 -- 172.21.15.37:0/298016626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c8075590 msgr2=0x7fa3c810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 --2- 172.21.15.37:0/298016626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c8075590 0x7fa3c810ef00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009990 tx=0x7fa3b4009ca0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 -- 172.21.15.37:0/298016626 shutdown_connections 2024-04-26T01:58:19.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 --2- 172.21.15.37:0/298016626 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c810f440 0x7fa3c8111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 --2- 172.21.15.37:0/298016626 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8074c70 0x7fa3c8075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 --2- 172.21.15.37:0/298016626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c8075590 0x7fa3c810ef00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 -- 172.21.15.37:0/298016626 >> 172.21.15.37:0/298016626 conn(0x7fa3c8079ba0 msgr2=0x7fa3c8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 -- 172.21.15.37:0/298016626 shutdown_connections 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.025+0000 7fa3cf200700 1 -- 172.21.15.37:0/298016626 wait complete. 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 Processor -- start 2024-04-26T01:58:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 -- start start 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 0x7fa3c819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8075590 0x7fa3c819b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c810f440 0x7fa3c819fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8114e70 con 0x7fa3c810f440 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8114cf0 con 0x7fa3c8074c70 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.026+0000 7fa3cf200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8114ff0 con 0x7fa3c8075590 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 0x7fa3c819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3cd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c810f440 0x7fa3c819fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8075590 0x7fa3c819b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 0x7fa3c819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:53936/0 (socket says 172.21.15.37:53936) 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 -- 172.21.15.37:0/4145030220 learned_addr learned my addr 172.21.15.37:0/4145030220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 -- 172.21.15.37:0/4145030220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8075590 msgr2=0x7fa3c819b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8075590 0x7fa3c819b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 -- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c810f440 msgr2=0x7fa3c819fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c810f440 0x7fa3c819fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7e00700 1 -- 172.21.15.37:0/4145030220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c81a0380 con 0x7fa3c8074c70 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.027+0000 7fa3c7400700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8075590 0x7fa3c819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.028+0000 7fa3c7e00700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 0x7fa3c819b3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800e9c0 tx=0x7fa3b800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.028+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800cbd0 con 0x7fa3c8074c70 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.028+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c81a0670 con 0x7fa3c8074c70 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.028+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b800ee60 con 0x7fa3c8074c70 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.028+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80055c0 con 0x7fa3c8074c70 2024-04-26T01:58:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.028+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c81a7ef0 con 0x7fa3c8074c70 2024-04-26T01:58:19.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.029+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c8069730 con 0x7fa3c8074c70 2024-04-26T01:58:19.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.031+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fa3b8004750 con 0x7fa3c8074c70 2024-04-26T01:58:19.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.032+0000 7fa3c4c00700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa3ac0a3ed0 0x7fa3ac0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:19.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.033+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa3b80d0d80 con 0x7fa3c8074c70 2024-04-26T01:58:19.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.033+0000 7fa3c7400700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa3ac0a3ed0 0x7fa3ac0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:19.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.033+0000 7fa3c7400700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa3ac0a3ed0 0x7fa3ac0a6390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8102640 tx=0x7fa3b401a040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:19.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.037+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3b809d1a0 con 0x7fa3c8074c70 2024-04-26T01:58:19.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.317+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3c802df80 con 0x7fa3c8074c70 2024-04-26T01:58:19.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.319+0000 7fa3c4c00700 1 -- 172.21.15.37:0/4145030220 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fa3b809caa0 con 0x7fa3c8074c70 2024-04-26T01:58:19.318 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:19.318 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:58:19.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa3ac0a3ed0 msgr2=0x7fa3ac0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa3ac0a3ed0 0x7fa3ac0a6390 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8102640 tx=0x7fa3b401a040 comp rx=0 tx=0).stop 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 msgr2=0x7fa3c819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 0x7fa3c819b3c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800e9c0 tx=0x7fa3b800ecd0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 shutdown_connections 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c810f440 0x7fa3c819fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fa3ac0a3ed0 0x7fa3ac0a6390 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa3c8074c70 0x7fa3c819b3c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 --2- 172.21.15.37:0/4145030220 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa3c8075590 0x7fa3c819b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 >> 172.21.15.37:0/4145030220 conn(0x7fa3c8079ba0 msgr2=0x7fa3c80ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 shutdown_connections 2024-04-26T01:58:19.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:19.322+0000 7fa3cf200700 1 -- 172.21.15.37:0/4145030220 wait complete. 2024-04-26T01:58:19.322 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:58:20.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:19 smithi045 ceph-mon[32520]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:58:20.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:19 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/4145030220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:20.151 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:19 smithi037 ceph-mon[27906]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:58:20.151 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:19 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/4145030220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:19 smithi071 ceph-mon[32578]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-04-26T01:58:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:19 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/4145030220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:20.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T01:58:20.713 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:22.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:21 smithi045 ceph-mon[32520]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-26T01:58:22.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:21 smithi071 ceph-mon[32578]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-26T01:58:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:21 smithi037 ceph-mon[27906]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-26T01:58:23.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 -- 172.21.15.37:0/1803002222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406a750 msgr2=0x7f311406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 --2- 172.21.15.37:0/1803002222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406a750 0x7f311406abb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3100009990 tx=0x7f3100009ca0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 -- 172.21.15.37:0/1803002222 shutdown_connections 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 --2- 172.21.15.37:0/1803002222 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311406b0f0 0x7f311410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 --2- 172.21.15.37:0/1803002222 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3114108a30 0x7f3114108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 --2- 172.21.15.37:0/1803002222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406a750 0x7f311406abb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 -- 172.21.15.37:0/1803002222 >> 172.21.15.37:0/1803002222 conn(0x7f31140788f0 msgr2=0x7f3114078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 -- 172.21.15.37:0/1803002222 shutdown_connections 2024-04-26T01:58:23.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.133+0000 7f311a200700 1 -- 172.21.15.37:0/1803002222 wait complete. 2024-04-26T01:58:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.134+0000 7f311a200700 1 Processor -- start 2024-04-26T01:58:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.134+0000 7f311a200700 1 -- start start 2024-04-26T01:58:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f311a200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 0x7f311419b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f311a200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406b0f0 0x7f311419b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f311a200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3114108a30 0x7f311419fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:23.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f311a200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3114112bd0 con 0x7f311406b0f0 2024-04-26T01:58:23.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f311a200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3114112a50 con 0x7f311406a750 2024-04-26T01:58:23.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f311a200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3114112d50 con 0x7f3114108a30 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f3113400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 0x7f311419b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f3112a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406b0f0 0x7f311419b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f3113400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 0x7f311419b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56170/0 (socket says 172.21.15.37:56170) 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f3113400700 1 -- 172.21.15.37:0/2779265434 learned_addr learned my addr 172.21.15.37:0/2779265434 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.135+0000 7f3113e00700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3114108a30 0x7f311419fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f3113400700 1 -- 172.21.15.37:0/2779265434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3114108a30 msgr2=0x7f311419fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f3113400700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3114108a30 0x7f311419fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f3113400700 1 -- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406b0f0 msgr2=0x7f311419b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:23.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f3113400700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406b0f0 0x7f311419b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f3113400700 1 -- 172.21.15.37:0/2779265434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31141a02f0 con 0x7f311406a750 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f3113400700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 0x7f311419b2e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f310400eb30 tx=0x7f310400ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310400cbd0 con 0x7f311406a750 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31141a05e0 con 0x7f311406a750 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31040044d0 con 0x7f311406a750 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.136+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3104003e30 con 0x7f311406a750 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.137+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31141a7e60 con 0x7f311406a750 2024-04-26T01:58:23.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.138+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3114069730 con 0x7f311406a750 2024-04-26T01:58:23.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.140+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f3104004000 con 0x7f311406a750 2024-04-26T01:58:23.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.141+0000 7f310be00700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f30f80a3ed0 0x7f30f80a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.141+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f31040d0ce0 con 0x7f311406a750 2024-04-26T01:58:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.142+0000 7f3112a00700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f30f80a3ed0 0x7f30f80a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:23.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.142+0000 7f3112a00700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f30f80a3ed0 0x7f30f80a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31000098e0 tx=0x7f3100005960 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:23.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.144+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31040042f0 con 0x7f311406a750 2024-04-26T01:58:23.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.376+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f311402df80 con 0x7f311406a750 2024-04-26T01:58:23.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.378+0000 7f310be00700 1 -- 172.21.15.37:0/2779265434 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+22871 (secure 0 0 0) 0x7f310409d1d0 con 0x7f311406a750 2024-04-26T01:58:23.377 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:23.377 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":92,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T01:57:56.610609+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4056686696":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/793487795":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/29145428":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/457714458":"2024-04-27T01:57:56.610509+0000","172.21.15.37:6800/691602839":"2024-04-27T01:57:56.610509+0000","172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/691602839":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T01:58:23.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f30f80a3ed0 msgr2=0x7f30f80a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f30f80a3ed0 0x7f30f80a6390 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f31000098e0 tx=0x7f3100005960 comp rx=0 tx=0).stop 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 msgr2=0x7f311419b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 0x7f311419b2e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f310400eb30 tx=0x7f310400ee40 comp rx=0 tx=0).stop 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 shutdown_connections 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f30f80a3ed0 0x7f30f80a6390 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3114108a30 0x7f311419fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311406a750 0x7f311419b2e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 --2- 172.21.15.37:0/2779265434 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311406b0f0 0x7f311419b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 >> 172.21.15.37:0/2779265434 conn(0x7f31140788f0 msgr2=0x7f311410df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 shutdown_connections 2024-04-26T01:58:23.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:23.382+0000 7f311a200700 1 -- 172.21.15.37:0/2779265434 wait complete. 2024-04-26T01:58:24.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:23 smithi045 ceph-mon[32520]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T01:58:24.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:23 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2779265434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:58:24.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:23 smithi071 ceph-mon[32578]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T01:58:24.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:23 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2779265434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:58:24.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:23 smithi037 ceph-mon[27906]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T01:58:24.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:23 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2779265434' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T01:58:24.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-26T01:58:24.918 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 -- 172.21.15.37:0/1254461643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c103850 msgr2=0x7f927c10bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 --2- 172.21.15.37:0/1254461643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c103850 0x7f927c10bd30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9264009900 tx=0x7f926402a480 comp rx=0 tx=0).stop 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 -- 172.21.15.37:0/1254461643 shutdown_connections 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 --2- 172.21.15.37:0/1254461643 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c10c350 0x7f927c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 --2- 172.21.15.37:0/1254461643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c103850 0x7f927c10bd30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 --2- 172.21.15.37:0/1254461643 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f927c102f30 0x7f927c103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 -- 172.21.15.37:0/1254461643 >> 172.21.15.37:0/1254461643 conn(0x7f927c0fc730 msgr2=0x7f927c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 -- 172.21.15.37:0/1254461643 shutdown_connections 2024-04-26T01:58:25.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.966+0000 7f9280a00640 1 -- 172.21.15.37:0/1254461643 wait complete. 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.967+0000 7f9280a00640 1 Processor -- start 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.967+0000 7f9280a00640 1 -- start start 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9280a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f927c102f30 0x7f927c19a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9280a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 0x7f927c19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9280a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c10c350 0x7f927c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9280a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f927c112030 con 0x7f927c10c350 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9280a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f927c111eb0 con 0x7f927c103850 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9280a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1121b0 con 0x7f927c102f30 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9271e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 0x7f927c19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f927b200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c10c350 0x7f927c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.968 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f927a800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f927c102f30 0x7f927c19a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9271e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 0x7f927c19acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56176/0 (socket says 172.21.15.37:56176) 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.968+0000 7f9271e00640 1 -- 172.21.15.37:0/2544167748 learned_addr learned my addr 172.21.15.37:0/2544167748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9271e00640 1 -- 172.21.15.37:0/2544167748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f927c102f30 msgr2=0x7f927c19a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9271e00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f927c102f30 0x7f927c19a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9271e00640 1 -- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c10c350 msgr2=0x7f927c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9271e00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c10c350 0x7f927c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9271e00640 1 -- 172.21.15.37:0/2544167748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f927c19f750 con 0x7f927c103850 2024-04-26T01:58:25.969 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9271e00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 0x7f927c19acf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9264005ab0 tx=0x7f9264003b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9273e00640 1 -- 172.21.15.37:0/2544167748 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926403f070 con 0x7f927c103850 2024-04-26T01:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f927c19f9e0 con 0x7f927c103850 2024-04-26T01:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.969+0000 7f9273e00640 1 -- 172.21.15.37:0/2544167748 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9264031b50 con 0x7f927c103850 2024-04-26T01:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.970+0000 7f9273e00640 1 -- 172.21.15.37:0/2544167748 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9264035030 con 0x7f927c103850 2024-04-26T01:58:25.970 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.970+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f927c1a72d0 con 0x7f927c103850 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 Processor -- start 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f9273e00640 1 -- 172.21.15.37:0/2544167748 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f9264042070 con 0x7f927c103850 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 -- start start 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454b9b0 0x55fb4454bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb44556650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb44558f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fb444b1900 con 0x55fb44556b90 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb44457ef0 con 0x55fb4454c2d0 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.973+0000 7f928418f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fb444ad8c0 con 0x55fb4454b9b0 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f9271e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb44558f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927b200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454b9b0 0x55fb4454bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f9271e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb44558f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50536/0 (socket says 172.21.15.37:50536) 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f9271e00640 1 -- 172.21.15.37:0/1861803271 learned_addr learned my addr 172.21.15.37:0/1861803271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927a800640 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb44556650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927a800640 1 -- 172.21.15.37:0/1861803271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454b9b0 msgr2=0x55fb4454bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927a800640 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454b9b0 0x55fb4454bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927a800640 1 -- 172.21.15.37:0/1861803271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 msgr2=0x55fb44558f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927a800640 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb44558f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.974+0000 7f927a800640 1 -- 172.21.15.37:0/1861803271 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb441c4210 con 0x55fb4454c2d0 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f927a800640 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb44556650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f926c009c00 tx=0x7f926c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=428a92d22531318e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f9273e00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f92600a3e20 0x7f92600a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f926b400640 1 -- 172.21.15.37:0/1861803271 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004230 con 0x55fb4454c2d0 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f926b400640 1 -- 172.21.15.37:0/1861803271 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c0043d0 con 0x55fb4454c2d0 2024-04-26T01:58:25.979 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f926b400640 1 -- 172.21.15.37:0/1861803271 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c004c70 con 0x55fb4454c2d0 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f9273e00640 1 -- 172.21.15.37:0/2544167748 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f92640f0ef0 con 0x7f927c103850 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.975+0000 7f927b200640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f92600a3e20 0x7f92600a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 -- 172.21.15.37:0/1861803271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 msgr2=0x55fb44556650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb44556650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f926c009c00 tx=0x7f926c011810 comp rx=0 tx=0).stop 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f927b200640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f92600a3e20 0x7f92600a62e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9274005fd0 tx=0x7f9274009500 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 -- 172.21.15.37:0/1861803271 shutdown_connections 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb44558f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb44556650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 --2- 172.21.15.37:0/1861803271 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454b9b0 0x55fb4454bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 -- 172.21.15.37:0/1861803271 >> 172.21.15.37:0/1861803271 conn(0x55fb44547ec0 msgr2=0x55fb445482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 -- 172.21.15.37:0/1861803271 shutdown_connections 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 -- 172.21.15.37:0/1861803271 wait complete. 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.976+0000 7f928418f740 1 -- start start 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 Processor -- start 2024-04-26T01:58:25.980 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44553b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb44554060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb44556b90 0x55fb445545a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fb445597c0 con 0x55fb4454b9b0 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55fb4446b310 con 0x55fb44556b90 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55fb44559940 con 0x55fb4454c2d0 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.977+0000 7f928418f740 10 client.0 _ll_register_callbacks cb 0x55fb44439cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44553b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44553b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50538/0 (socket says 172.21.15.37:50538) 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 -- 172.21.15.37:0/1962992962 learned_addr learned my addr 172.21.15.37:0/1962992962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f9271e00640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb44554060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927b200640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb44556b90 0x55fb445545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 -- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 msgr2=0x55fb44554060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb44554060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 -- 172.21.15.37:0/1962992962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb44556b90 msgr2=0x55fb445545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb44556b90 0x55fb445545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 -- 172.21.15.37:0/1962992962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb44290620 con 0x55fb4454b9b0 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f927a800640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44553b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f926c019ac0 tx=0x7f926c019af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd51df203f14305 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.978+0000 7f925fe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f925fe00640 1 -- 172.21.15.37:0/1962992962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c028020 con 0x55fb4454b9b0 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f925fe00640 1 -- 172.21.15.37:0/1962992962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f926c024620 con 0x55fb4454b9b0 2024-04-26T01:58:25.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c0043d0 con 0x55fb4454b9b0 2024-04-26T01:58:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f925fe00640 1 -- 172.21.15.37:0/1962992962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f926c033050 con 0x55fb4454b9b0 2024-04-26T01:58:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f928418f740 10 client.15009 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55fb44565a20 con 0x55fb4454b9b0 2024-04-26T01:58:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.979+0000 7f925fe00640 1 -- 172.21.15.37:0/1962992962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c0043d0 con 0x55fb4454b9b0 2024-04-26T01:58:25.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f925fe00640 1 -- 172.21.15.37:0/1962992962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f926c018070 con 0x55fb4454b9b0 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f928418f740 10 client.15009 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f928418f740 10 client.15009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f928418f740 20 client.15009 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f928418f740 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb44566e20 0x55fb44569210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f928418f740 4 client.15009 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb444c58b0 con 0x55fb44566e20 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.980+0000 7f9271e00640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb44566e20 0x55fb44569210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.981+0000 7f9271e00640 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb44566e20 0x55fb44569210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cd5b08f8f20d3fc server_cookie=df59eb2513559eef in_seq=0 out_seq=0 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.981+0000 7f925fe00640 10 client.15009 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.981+0000 7f925fe00640 1 -- 172.21.15.37:0/1962992962 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb444c58b0 con 0x55fb44566e20 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.981+0000 7f925fe00640 10 client.15009 handle_command_reply: tid=0 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.981+0000 7f928418f740 1 client.15009 shutdown 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 msgr2=0x55fb44553b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44553b20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f926c019ac0 tx=0x7f926c019af0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 shutdown_connections 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb44566e20 0x55fb44569210 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb44556b90 0x55fb445545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb44554060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 --2- 172.21.15.37:0/1962992962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44553b20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 >> 172.21.15.37:0/1962992962 conn(0x55fb44547ec0 msgr2=0x55fb4454dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f925fe00640 0 client.15009 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 shutdown_connections 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 1 -- 172.21.15.37:0/1962992962 wait complete. 2024-04-26T01:58:25.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.982+0000 7f928418f740 20 client.15009 trim_cache size 0 max 16384 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.998+0000 7f928418f740 1 Processor -- start 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.998+0000 7f928418f740 1 -- start start 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f928418f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44562680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f928418f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb4456e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f928418f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb44556b90 0x55fb4456eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f928418f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1adae0 con 0x55fb4454b9b0 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f928418f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1b0360 con 0x55fb4454c2d0 2024-04-26T01:58:25.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f928418f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1ae150 con 0x55fb44556b90 2024-04-26T01:58:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f927a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44562680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:25.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f927a800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44562680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50550/0 (socket says 172.21.15.37:50550) 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f927a800640 1 -- 172.21.15.37:0/770381417 learned_addr learned my addr 172.21.15.37:0/770381417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f9271e00640 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb4456e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:25.999+0000 7f927b200640 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb44556b90 0x55fb4456eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f927a800640 1 -- 172.21.15.37:0/770381417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb44556b90 msgr2=0x55fb4456eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f927a800640 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb44556b90 0x55fb4456eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f927a800640 1 -- 172.21.15.37:0/770381417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 msgr2=0x55fb4456e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f927a800640 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb4456e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f927a800640 1 -- 172.21.15.37:0/770381417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb44569770 con 0x55fb4454b9b0 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f927a800640 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44562680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f926c024ef0 tx=0x7f926c019be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bac79b18222c8e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f926aa00640 1 -- 172.21.15.37:0/770381417 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c028040 con 0x55fb4454b9b0 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f926aa00640 1 -- 172.21.15.37:0/770381417 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c01f040 con 0x55fb4454b9b0 2024-04-26T01:58:26.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.000+0000 7f926aa00640 1 -- 172.21.15.37:0/770381417 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c018b00 con 0x55fb4454b9b0 2024-04-26T01:58:26.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 -- 172.21.15.37:0/770381417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 msgr2=0x55fb44562680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44562680 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f926c024ef0 tx=0x7f926c019be0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 -- 172.21.15.37:0/770381417 shutdown_connections 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb44556b90 0x55fb4456eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454c2d0 0x55fb4456e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 --2- 172.21.15.37:0/770381417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb4454b9b0 0x55fb44562680 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 -- 172.21.15.37:0/770381417 >> 172.21.15.37:0/770381417 conn(0x55fb44547ec0 msgr2=0x55fb4454edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 -- 172.21.15.37:0/770381417 shutdown_connections 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 -- 172.21.15.37:0/770381417 wait complete. 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.001+0000 7f928418f740 1 -- start start 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 Processor -- start 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 0x55fb4455ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb4455c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:26.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb4455c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1ae820 con 0x55fb44556b90 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1afc90 con 0x55fb4454b9b0 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f927c1aeef0 con 0x55fb4454c2d0 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f928418f740 10 client.0 _ll_register_callbacks cb 0x7f926c009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.002+0000 7f927a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 0x55fb4455ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 0x55fb4455ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56196/0 (socket says 172.21.15.37:56196) 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 -- 172.21.15.37:0/1060550277 learned_addr learned my addr 172.21.15.37:0/1060550277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927b200640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb4455c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f9271e00640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb4455c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 -- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 msgr2=0x55fb4455c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb4455c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 -- 172.21.15.37:0/1060550277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 msgr2=0x55fb4455c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb4455c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 -- 172.21.15.37:0/1060550277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb4455cc40 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f927a800640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 0x55fb4455ed30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f926c019b30 tx=0x7f926c009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af614291773073a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f925fe00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.003+0000 7f925fe00640 1 -- 172.21.15.37:0/1060550277 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c028070 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f925fe00640 1 -- 172.21.15.37:0/1060550277 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f926c005520 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f926c01f040 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f925fe00640 1 -- 172.21.15.37:0/1060550277 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f926c045050 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f928418f740 10 client.25106 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55fb445ccfa0 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f925fe00640 1 -- 172.21.15.37:0/1060550277 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f926c01f040 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f925fe00640 1 -- 172.21.15.37:0/1060550277 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f926c024590 con 0x55fb4454b9b0 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.004+0000 7f928418f740 10 client.25106 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f928418f740 10 client.25106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f928418f740 20 client.25106 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:26.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f928418f740 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb445cd650 0x55fb445cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f928418f740 4 client.25106 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fb444bf320 con 0x55fb445cd650 2024-04-26T01:58:26.007 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:25 smithi045 ceph-mon[32520]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T01:58:26.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f9271e00640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb445cd650 0x55fb445cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f9271e00640 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb445cd650 0x55fb445cfa40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c55cc97987eec48e server_cookie=f6e5d29e7939ccb5 in_seq=0 out_seq=0 2024-04-26T01:58:26.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.005+0000 7f925fe00640 10 client.25106 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f925fe00640 1 -- 172.21.15.37:0/1060550277 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fb444bf320 con 0x55fb445cd650 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f925fe00640 10 client.25106 handle_command_reply: tid=0 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 client.25106 shutdown 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 msgr2=0x55fb4455ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 0x55fb4455ed30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f926c019b30 tx=0x7f926c009260 comp rx=0 tx=0).stop 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 shutdown_connections 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55fb445cd650 0x55fb445cfa40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fb44556b90 0x55fb4455c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55fb4454c2d0 0x55fb4455c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 --2- 172.21.15.37:0/1060550277 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55fb4454b9b0 0x55fb4455ed30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f925fe00640 0 client.25106 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.009+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 >> 172.21.15.37:0/1060550277 conn(0x55fb44547ec0 msgr2=0x55fb4454d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.010+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 shutdown_connections 2024-04-26T01:58:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.010+0000 7f928418f740 1 -- 172.21.15.37:0/1060550277 wait complete. 2024-04-26T01:58:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.010+0000 7f928418f740 20 client.25106 trim_cache size 0 max 16384 2024-04-26T01:58:26.010 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:25 smithi037 ceph-mon[27906]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T01:58:26.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.010+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f92600a3e20 msgr2=0x7f92600a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f92600a3e20 0x7f92600a62e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9274005fd0 tx=0x7f9274009500 comp rx=0 tx=0).stop 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 msgr2=0x7f927c19acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 0x7f927c19acf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9264005ab0 tx=0x7f9264003b60 comp rx=0 tx=0).stop 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 shutdown_connections 2024-04-26T01:58:26.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f92600a3e20 0x7f92600a62e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f927c10c350 0x7f927c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f927c103850 0x7f927c19acf0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 --2- 172.21.15.37:0/2544167748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f927c102f30 0x7f927c19a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 >> 172.21.15.37:0/2544167748 conn(0x7f927c0fc730 msgr2=0x7f927c101c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 shutdown_connections 2024-04-26T01:58:26.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:26.011+0000 7f9280a00640 1 -- 172.21.15.37:0/2544167748 wait complete. 2024-04-26T01:58:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:25 smithi071 ceph-mon[32578]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-26T01:58:28.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:27 smithi045 ceph-mon[32520]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-26T01:58:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:27 smithi071 ceph-mon[32578]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-26T01:58:28.188 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:27 smithi037 ceph-mon[27906]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-04-26T01:58:28.265 INFO:teuthology.orchestra.run.smithi037.stdout:{"run_dir":"/var/run/ceph"} 2024-04-26T01:58:28.266 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-26T01:58:28.266 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3448 2024-04-26T01:58:28.266 DEBUG:tasks.quiescer:Yielding 2024-04-26T01:58:28.266 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T01:58:28.280 INFO:tasks.workunit:Pulling workunits from ref afa1933b0fdbb6c99c947d1eda34d661d23cd327 2024-04-26T01:58:28.281 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T01:58:28.281 INFO:tasks.workunit:timeout=3h 2024-04-26T01:58:28.281 INFO:tasks.workunit:cleanup=True 2024-04-26T01:58:28.281 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T01:58:28.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:58:28.336 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-04-26 01:54:50.035933103 +0000 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-04-26 01:54:50.035933103 +0000 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-04-26 01:57:55.578399162 +0000 2024-04-26T01:58:28.337 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-04-26 01:54:50.035933103 +0000 2024-04-26T01:58:28.337 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T01:58:28.337 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T01:58:28.474 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout afa1933b0fdbb6c99c947d1eda34d661d23cd327 2024-04-26T01:58:28.533 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T01:58:28.736 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:30.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:29 smithi045 ceph-mon[32520]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-26T01:58:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:29 smithi071 ceph-mon[32578]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-26T01:58:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:29 smithi037 ceph-mon[27906]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-04-26T01:58:31.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 -- 172.21.15.37:0/1983775134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 msgr2=0x7ff1f0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:31.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 --2- 172.21.15.37:0/1983775134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f0102f10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc009990 tx=0x7ff1dc009ca0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 -- 172.21.15.37:0/1983775134 shutdown_connections 2024-04-26T01:58:31.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 --2- 172.21.15.37:0/1983775134 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 0x7ff1f010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 --2- 172.21.15.37:0/1983775134 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 0x7ff1f0108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 --2- 172.21.15.37:0/1983775134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f0102f10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.277+0000 7ff1f7000700 1 -- 172.21.15.37:0/1983775134 >> 172.21.15.37:0/1983775134 conn(0x7ff1f0079bf0 msgr2=0x7ff1f007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.278+0000 7ff1f7000700 1 -- 172.21.15.37:0/1983775134 shutdown_connections 2024-04-26T01:58:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.278+0000 7ff1f7000700 1 -- 172.21.15.37:0/1983775134 wait complete. 2024-04-26T01:58:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.279+0000 7ff1f7000700 1 Processor -- start 2024-04-26T01:58:31.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.279+0000 7ff1f7000700 1 -- start start 2024-04-26T01:58:31.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.279+0000 7ff1f7000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f019b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:31.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f7000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 0x7ff1f019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:31.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f7000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 0x7ff1f019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:31.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f7000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0112c10 con 0x7ff1f0102a90 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f7000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0112a90 con 0x7ff1f0103450 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f7000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f0112d90 con 0x7ff1f0108a80 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f019b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f019b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44780/0 (socket says 172.21.15.37:44780) 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 -- 172.21.15.37:0/3720541187 learned_addr learned my addr 172.21.15.37:0/3720541187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1ef400700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 0x7ff1f019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:31.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f5000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 0x7ff1f019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 -- 172.21.15.37:0/3720541187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 msgr2=0x7ff1f019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 0x7ff1f019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 -- 172.21.15.37:0/3720541187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 msgr2=0x7ff1f019b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 0x7ff1f019b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1efe00700 1 -- 172.21.15.37:0/3720541187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f01a0340 con 0x7ff1f0102a90 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1f5000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 0x7ff1f019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.280+0000 7ff1ef400700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 0x7ff1f019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.281+0000 7ff1efe00700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f019b330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e9a0 tx=0x7ff1e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.281+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000cbd0 con 0x7ff1f0102a90 2024-04-26T01:58:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.281+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1e000cd70 con 0x7ff1f0102a90 2024-04-26T01:58:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.281+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f01a0630 con 0x7ff1f0102a90 2024-04-26T01:58:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.281+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0018680 con 0x7ff1f0102a90 2024-04-26T01:58:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.281+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01a7eb0 con 0x7ff1f0102a90 2024-04-26T01:58:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.283+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0069730 con 0x7ff1f0102a90 2024-04-26T01:58:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.285+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff1e0010430 con 0x7ff1f0102a90 2024-04-26T01:58:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.287+0000 7ff1ecc00700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff1d40a3f50 0x7ff1d40a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.287+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff1e00d0c20 con 0x7ff1f0102a90 2024-04-26T01:58:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.287+0000 7ff1ef400700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff1d40a3f50 0x7ff1d40a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.288+0000 7ff1ef400700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff1d40a3f50 0x7ff1d40a6410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00b2e0 tx=0x7ff1dc01a040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.292+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff1e009d040 con 0x7ff1f0102a90 2024-04-26T01:58:31.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.585+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1f002df80 con 0x7ff1f0102a90 2024-04-26T01:58:31.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.587+0000 7ff1ecc00700 1 -- 172.21.15.37:0/3720541187 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7ff1e009c940 con 0x7ff1f0102a90 2024-04-26T01:58:31.586 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:31.586 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:58:31.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff1d40a3f50 msgr2=0x7ff1d40a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:31.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff1d40a3f50 0x7ff1d40a6410 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc00b2e0 tx=0x7ff1dc01a040 comp rx=0 tx=0).stop 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 msgr2=0x7ff1f019b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f019b330 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000e9a0 tx=0x7ff1e000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 shutdown_connections 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1f0102a90 0x7ff1f019b330 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff1d40a3f50 0x7ff1d40a6410 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff1f0108a80 0x7ff1f019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 --2- 172.21.15.37:0/3720541187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff1f0103450 0x7ff1f019b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 >> 172.21.15.37:0/3720541187 conn(0x7ff1f0079bf0 msgr2=0x7ff1f010dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 shutdown_connections 2024-04-26T01:58:31.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:31.590+0000 7ff1f7000700 1 -- 172.21.15.37:0/3720541187 wait complete. 2024-04-26T01:58:31.591 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:58:31.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:31 smithi037 ceph-mon[27906]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:58:31.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:31 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3720541187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:32.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:31 smithi071 ceph-mon[32578]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:58:32.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:31 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3720541187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:31 smithi045 ceph-mon[32520]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:58:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:31 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3720541187' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:32.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T01:54:12.162501+0000', 'modified': '2024-04-26T01:54:33.402956+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24514}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24509': {'gid': 24509, 'name': 'b', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.45:6839/60713297', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.45:6838', 'nonce': 60713297}, {'type': 'v1', 'addr': '172.21.15.45:6839', 'nonce': 60713297}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24514': {'gid': 24514, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.71:6839/1148187051', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.71:6838', 'nonce': 1148187051}, {'type': 'v1', 'addr': '172.21.15.71:6839', 'nonce': 1148187051}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24514, 'qdb_cluster': [24514]} 2024-04-26T01:58:32.673 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-26T01:58:32.673 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T01:58:33.128 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:33.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:33 smithi037 ceph-mon[27906]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-26T01:58:34.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:33 smithi071 ceph-mon[32578]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-26T01:58:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:33 smithi045 ceph-mon[32520]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 938 B/s wr, 3 op/s 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 -- 172.21.15.37:0/4175648263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4109e90 msgr2=0x7f51a4111a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 --2- 172.21.15.37:0/4175648263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4109e90 0x7f51a4111a60 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f519c00b0d0 tx=0x7f519c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 -- 172.21.15.37:0/4175648263 shutdown_connections 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 --2- 172.21.15.37:0/4175648263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4109e90 0x7f51a4111a60 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 --2- 172.21.15.37:0/4175648263 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4104db0 0x7f51a4105190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 --2- 172.21.15.37:0/4175648263 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4105760 0x7f51a41097b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 -- 172.21.15.37:0/4175648263 >> 172.21.15.37:0/4175648263 conn(0x7f51a4100620 msgr2=0x7f51a4102a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 -- 172.21.15.37:0/4175648263 shutdown_connections 2024-04-26T01:58:35.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.557+0000 7f51a9c00700 1 -- 172.21.15.37:0/4175648263 wait complete. 2024-04-26T01:58:35.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.558+0000 7f51a9c00700 1 Processor -- start 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.558+0000 7f51a9c00700 1 -- start start 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4104db0 0x7f51a419b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9c00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 0x7f51a419b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 0x7f51a419faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9c00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4114d40 con 0x7f51a4104db0 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9c00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4114bc0 con 0x7f51a4105760 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9c00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4114ec0 con 0x7f51a4109e90 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 0x7f51a419faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 0x7f51a419b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 0x7f51a419faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34556/0 (socket says 172.21.15.37:34556) 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a9200700 1 -- 172.21.15.37:0/246583852 learned_addr learned my addr 172.21.15.37:0/246583852 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.559+0000 7f51a3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 0x7f51a419b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:49604/0 (socket says 172.21.15.37:49604) 2024-04-26T01:58:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a3e00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4104db0 0x7f51a419b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a9200700 1 -- 172.21.15.37:0/246583852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 msgr2=0x7f51a419b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a9200700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 0x7f51a419b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a9200700 1 -- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4104db0 msgr2=0x7f51a419b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a9200700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4104db0 0x7f51a419b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a9200700 1 -- 172.21.15.37:0/246583852 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a41a01e0 con 0x7f51a4109e90 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a3e00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4104db0 0x7f51a419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a3400700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 0x7f51a419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a9200700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 0x7f51a419faa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f519c007a00 tx=0x7f519c007a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.560+0000 7f51a0c00700 1 -- 172.21.15.37:0/246583852 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c01f070 con 0x7f51a4109e90 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.561+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41a0470 con 0x7f51a4109e90 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.561+0000 7f51a0c00700 1 -- 172.21.15.37:0/246583852 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f519c004c50 con 0x7f51a4109e90 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.561+0000 7f51a0c00700 1 -- 172.21.15.37:0/246583852 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c01b690 con 0x7f51a4109e90 2024-04-26T01:58:35.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.561+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41a7d50 con 0x7f51a4109e90 2024-04-26T01:58:35.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.564+0000 7f51a0c00700 1 -- 172.21.15.37:0/246583852 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f519c022070 con 0x7f51a4109e90 2024-04-26T01:58:35.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 Processor -- start 2024-04-26T01:58:35.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 -- start start 2024-04-26T01:58:35.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959967c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 0x55a1959a1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959a1c40 0x55a1959a4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1958e9e40 con 0x55a1959963e0 2024-04-26T01:58:35.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a1958906e0 con 0x55a195996d00 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.566+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a1958e5e00 con 0x55a1959a1c40 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a0c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f51880a3ed0 0x7f51880a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 0x55a1959a1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 0x55a1959a1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:49608/0 (socket says 172.21.15.37:49608) 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 -- 172.21.15.37:0/173810727 learned_addr learned my addr 172.21.15.37:0/173810727 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a0c00700 1 -- 172.21.15.37:0/246583852 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f519c00f020 con 0x7f51a4109e90 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3400700 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959a1c40 0x55a1959a4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3400700 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959967c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 -- 172.21.15.37:0/173810727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959a1c40 msgr2=0x55a1959a4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959a1c40 0x55a1959a4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a9200700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f51880a3ed0 0x7f51880a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 -- 172.21.15.37:0/173810727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 msgr2=0x55a1959967c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959967c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.567+0000 7f51a3e00700 1 -- 172.21.15.37:0/173810727 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a195886630 con 0x55a195996d00 2024-04-26T01:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.568+0000 7f51a3400700 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959a1c40 0x55a1959a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.568+0000 7f51a9200700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f51880a3ed0 0x7f51880a6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f519c0cf0d0 tx=0x7f519c00b3e0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.568+0000 7f51a3e00700 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 0x55a1959a1700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5194009cc0 tx=0x7f5194010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a3c7a89a8317d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.569+0000 7f5199600700 1 -- 172.21.15.37:0/173810727 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194010bc0 con 0x55a195996d00 2024-04-26T01:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.569+0000 7f5199600700 1 -- 172.21.15.37:0/173810727 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51940044e0 con 0x55a195996d00 2024-04-26T01:58:35.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.569+0000 7f5199600700 1 -- 172.21.15.37:0/173810727 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194012450 con 0x55a195996d00 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 -- 172.21.15.37:0/173810727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 msgr2=0x55a1959a1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 0x55a1959a1700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5194009cc0 tx=0x7f5194010370 comp rx=0 tx=0).stop 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 -- 172.21.15.37:0/173810727 shutdown_connections 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959967c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959a1c40 0x55a1959a4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/173810727 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a195996d00 0x55a1959a1700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 -- 172.21.15.37:0/173810727 >> 172.21.15.37:0/173810727 conn(0x55a1958c5550 msgr2=0x55a195992a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:35.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 -- 172.21.15.37:0/173810727 shutdown_connections 2024-04-26T01:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.570+0000 7f51b5aaab80 1 -- 172.21.15.37:0/173810727 wait complete. 2024-04-26T01:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.571+0000 7f51b5aaab80 1 -- start start 2024-04-26T01:58:35.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.571+0000 7f51b5aaab80 1 Processor -- start 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.571+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959963e0 0x55a1959a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 0x55a19599e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a19599eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1959a47c0 con 0x55a195996d00 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a1958a3810 con 0x55a1959a1c40 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a1959a4940 con 0x55a1959963e0 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51b5aaab80 10 client.0 _ll_register_callbacks cb 0x55a1958826d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 0x55a19599e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 0x55a19599e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44800/0 (socket says 172.21.15.37:44800) 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3e00700 1 -- 172.21.15.37:0/1187384023 learned_addr learned my addr 172.21.15.37:0/1187384023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:35.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3400700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959963e0 0x55a1959a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3400700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a19599eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3e00700 1 -- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959963e0 msgr2=0x55a1959a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3e00700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959963e0 0x55a1959a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.572+0000 7f51a3e00700 1 -- 172.21.15.37:0/1187384023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 msgr2=0x55a19599eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f51a3e00700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a19599eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f51a3e00700 1 -- 172.21.15.37:0/1187384023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a195837880 con 0x55a195996d00 2024-04-26T01:58:35.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f51a3400700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a19599eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f51a3e00700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 0x55a19599e650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5194003b30 tx=0x7f5194010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ab3db738178722 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:35.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f518e000700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f518e000700 1 -- 172.21.15.37:0/1187384023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194005240 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f518e000700 1 -- 172.21.15.37:0/1187384023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f51940053e0 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.573+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51940044e0 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f518e000700 1 -- 172.21.15.37:0/1187384023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f51940055e0 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f51b5aaab80 10 client.15030 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a195880aa0 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f518e000700 1 -- 172.21.15.37:0/1187384023 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51940044e0 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f518e000700 1 -- 172.21.15.37:0/1187384023 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f519400f040 con 0x55a195996d00 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f51b5aaab80 10 client.15030 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.574+0000 7f51b5aaab80 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.575+0000 7f51b5aaab80 20 client.15030 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.575+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a1959b9a30 0x55a1959baea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.575+0000 7f51b5aaab80 4 client.15030 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.575+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1958f7630 con 0x55a1959b9a30 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.575+0000 7f51a3400700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a1959b9a30 0x55a1959baea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.576+0000 7f51a3400700 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a1959b9a30 0x55a1959baea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34081c4230ceeb65 server_cookie=4f36311fa0cea974 in_seq=0 out_seq=0 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.576+0000 7f518e000700 10 client.15030 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f518e000700 1 -- 172.21.15.37:0/1187384023 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1958f7630 con 0x55a1959b9a30 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f518e000700 10 client.15030 handle_command_reply: tid=0 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 client.15030 shutdown 2024-04-26T01:58:35.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 msgr2=0x55a19599e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 0x55a19599e650 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f5194003b30 tx=0x7f5194010bf0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 shutdown_connections 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a1959b9a30 0x55a1959baea0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a19599eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a1959963e0 0x55a1959a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/1187384023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a195996d00 0x55a19599e650 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 >> 172.21.15.37:0/1187384023 conn(0x55a1958c5550 msgr2=0x55a195992680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f518e000700 0 client.15030 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.577+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 shutdown_connections 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.578+0000 7f51b5aaab80 1 -- 172.21.15.37:0/1187384023 wait complete. 2024-04-26T01:58:35.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.578+0000 7f51b5aaab80 20 client.15030 trim_cache size 0 max 16384 2024-04-26T01:58:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.622+0000 7f51b5aaab80 1 Processor -- start 2024-04-26T01:58:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 -- start start 2024-04-26T01:58:35.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a195992b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a195999db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959a1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1959b91e0 con 0x55a1959963e0 2024-04-26T01:58:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a1959b98b0 con 0x55a1959a1c40 2024-04-26T01:58:35.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a1959b9f80 con 0x55a195996d00 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.623+0000 7f51a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a195992b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a195992b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44808/0 (socket says 172.21.15.37:44808) 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3400700 1 -- 172.21.15.37:0/3248164423 learned_addr learned my addr 172.21.15.37:0/3248164423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a195999db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3400700 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959a1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 -- 172.21.15.37:0/3248164423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 msgr2=0x55a1959a1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959a1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 -- 172.21.15.37:0/3248164423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 msgr2=0x55a195992b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a195992b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 -- 172.21.15.37:0/3248164423 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a195996b00 con 0x55a195996d00 2024-04-26T01:58:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.624+0000 7f51a3e00700 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a195999db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5194000c00 tx=0x7f5194004660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf630eb7a366bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.625+0000 7f5198c00700 1 -- 172.21.15.37:0/3248164423 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194032070 con 0x55a195996d00 2024-04-26T01:58:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.625+0000 7f5198c00700 1 -- 172.21.15.37:0/3248164423 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51940046d0 con 0x55a195996d00 2024-04-26T01:58:35.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.625+0000 7f5198c00700 1 -- 172.21.15.37:0/3248164423 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194005440 con 0x55a195996d00 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 -- 172.21.15.37:0/3248164423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 msgr2=0x55a195999db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a195999db0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5194000c00 tx=0x7f5194004660 comp rx=0 tx=0).stop 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 -- 172.21.15.37:0/3248164423 shutdown_connections 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a195992b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959a1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/3248164423 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a195999db0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 -- 172.21.15.37:0/3248164423 >> 172.21.15.37:0/3248164423 conn(0x55a1958c5550 msgr2=0x55a1959996a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 -- 172.21.15.37:0/3248164423 shutdown_connections 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 -- 172.21.15.37:0/3248164423 wait complete. 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.626+0000 7f51b5aaab80 1 -- start start 2024-04-26T01:58:35.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 Processor -- start 2024-04-26T01:58:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959bdff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a1959be530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1959bad20 con 0x55a1959963e0 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55a1959ba650 con 0x55a1959a1c40 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55a1959bb3f0 con 0x55a195996d00 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.627+0000 7f51b5aaab80 10 client.0 _ll_register_callbacks cb 0x55a1958826d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959bdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a1959be530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959bdff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44820/0 (socket says 172.21.15.37:44820) 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 -- 172.21.15.37:0/2826293208 learned_addr learned my addr 172.21.15.37:0/2826293208 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 -- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 msgr2=0x55a1959be530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a1959be530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 -- 172.21.15.37:0/2826293208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 msgr2=0x55a1959c27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959c27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3400700 1 -- 172.21.15.37:0/2826293208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1959bedf0 con 0x55a1959963e0 2024-04-26T01:58:35.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.628+0000 7f51a3e00700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a1959be530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f51a3400700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959bdff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f519000a010 tx=0x7f5190013460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a009e3595234f413 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f518e000700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:58:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f518e000700 1 -- 172.21.15.37:0/2826293208 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190013110 con 0x55a1959963e0 2024-04-26T01:58:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f518e000700 1 -- 172.21.15.37:0/2826293208 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f519001c040 con 0x55a1959963e0 2024-04-26T01:58:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51940046d0 con 0x55a1959963e0 2024-04-26T01:58:35.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f518e000700 1 -- 172.21.15.37:0/2826293208 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5190039050 con 0x55a1959963e0 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f51b5aaab80 10 client.15036 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.629+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55a195a211f0 con 0x55a1959963e0 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f518e000700 1 -- 172.21.15.37:0/2826293208 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51940046d0 con 0x55a1959963e0 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f518e000700 1 -- 172.21.15.37:0/2826293208 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5190011560 con 0x55a1959963e0 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f51b5aaab80 10 client.15036 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f51b5aaab80 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f51b5aaab80 20 client.15036 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a195a25dd0 0x55a195a24ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f51b5aaab80 4 client.15036 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.630+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "status"}) -- 0x55a1958f16d0 con 0x55a195a25dd0 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.631+0000 7f51a3e00700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a195a25dd0 0x55a195a24ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.631+0000 7f518e000700 10 client.15036 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:35.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.631+0000 7f51a3e00700 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a195a25dd0 0x55a195a24ae0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a26f73eefc8d83e5 server_cookie=78bef6ac9f6f991c in_seq=0 out_seq=0 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.631+0000 7f518e000700 1 -- 172.21.15.37:0/2826293208 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55a1958f16d0 con 0x55a195a25dd0 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.631+0000 7f518e000700 10 client.15036 handle_command_reply: tid=0 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.631+0000 7f51b5aaab80 1 client.15036 shutdown 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 msgr2=0x55a1959bdff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959bdff0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f519000a010 tx=0x7f5190013460 comp rx=0 tx=0).stop 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 shutdown_connections 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55a195a25dd0 0x55a195a24ae0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1959963e0 0x55a1959bdff0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55a1959a1c40 0x55a1959c27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 --2- 172.21.15.37:0/2826293208 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55a195996d00 0x55a1959be530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f518e000700 0 client.15036 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 >> 172.21.15.37:0/2826293208 conn(0x55a1958c5550 msgr2=0x55a19599fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:35.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 shutdown_connections 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.632+0000 7f51b5aaab80 1 -- 172.21.15.37:0/2826293208 wait complete. 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.633+0000 7f51b5aaab80 20 client.15036 trim_cache size 0 max 16384 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "5d371fa8-036e-11ef-bc93-c7b262605968", 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24514, 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-04-26T01:58:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-04-26T01:58:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 262.61248587099999, 2024-04-26T01:58:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 19, 2024-04-26T01:58:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 92, 2024-04-26T01:58:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T01:58:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 299.73475830299998 2024-04-26T01:58:35.635 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.633+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f51880a3ed0 msgr2=0x7f51880a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.633+0000 7f51a9c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f51880a3ed0 0x7f51880a6390 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f519c0cf0d0 tx=0x7f519c00b3e0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.633+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 msgr2=0x7f51a419faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.633+0000 7f51a9c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 0x7f51a419faa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f519c007a00 tx=0x7f519c007a30 comp rx=0 tx=0).stop 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 shutdown_connections 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51a4104db0 0x7f51a419b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f51880a3ed0 0x7f51880a6390 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f51a4109e90 0x7f51a419faa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 --2- 172.21.15.37:0/246583852 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f51a4105760 0x7f51a419b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 >> 172.21.15.37:0/246583852 conn(0x7f51a4100620 msgr2=0x7f51a4101160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 shutdown_connections 2024-04-26T01:58:35.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:35.634+0000 7f51a9c00700 1 -- 172.21.15.37:0/246583852 wait complete. 2024-04-26T01:58:35.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:35 smithi037 ceph-mon[27906]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T01:58:36.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:35 smithi071 ceph-mon[32578]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T01:58:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:35 smithi045 ceph-mon[32520]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T01:58:36.613 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-26T01:58:38.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:37 smithi071 ceph-mon[32578]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T01:58:38.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:37 smithi037 ceph-mon[27906]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T01:58:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:37 smithi045 ceph-mon[32520]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1023 B/s wr, 2 op/s 2024-04-26T01:58:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 -- 172.21.15.37:0/551831195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 msgr2=0x7fbdd0102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/551831195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd0102590 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0009860 tx=0x7fbdc002a340 comp rx=0 tx=0).stop 2024-04-26T01:58:39.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 -- 172.21.15.37:0/551831195 shutdown_connections 2024-04-26T01:58:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/551831195 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 0x7fbdd010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/551831195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd0102b60 0x7fbdd0106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/551831195 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd0102590 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 -- 172.21.15.37:0/551831195 >> 172.21.15.37:0/551831195 conn(0x7fbdd00fda10 msgr2=0x7fbdd00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 -- 172.21.15.37:0/551831195 shutdown_connections 2024-04-26T01:58:39.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.152+0000 7fbdd6a00640 1 -- 172.21.15.37:0/551831195 wait complete. 2024-04-26T01:58:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 Processor -- start 2024-04-26T01:58:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 -- start start 2024-04-26T01:58:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 0x7fbdd01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd019a780 0x7fbdd019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01121c0 con 0x7fbdd019a780 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0112040 con 0x7fbdd0107280 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.153+0000 7fbdd6a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0112340 con 0x7fbdd01021b0 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdd4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd019a780 0x7fbdd019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:34622/0 (socket says 172.21.15.37:34622) 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2972191597 learned_addr learned my addr 172.21.15.37:0/2972191597 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcf400640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 0x7fbdd01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2972191597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 msgr2=0x7fbdd01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 0x7fbdd01a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd019a780 msgr2=0x7fbdd019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd019a780 0x7fbdd019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2972191597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd019b310 con 0x7fbdd01021b0 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdcf400640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 0x7fbdd01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.154+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd019a780 0x7fbdd019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.155+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd01a0610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0035040 tx=0x7fbdc0032d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.155+0000 7fbdccc00640 1 -- 172.21.15.37:0/2972191597 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0037040 con 0x7fbdd01021b0 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.155+0000 7fbdccc00640 1 -- 172.21.15.37:0/2972191597 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0041070 con 0x7fbdd01021b0 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.155+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd019b5a0 con 0x7fbdd01021b0 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.155+0000 7fbdccc00640 1 -- 172.21.15.37:0/2972191597 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0003850 con 0x7fbdd01021b0 2024-04-26T01:58:39.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.155+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd01a76f0 con 0x7fbdd01021b0 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.158+0000 7fbdd998d740 1 Processor -- start 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 -- start start 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f779839c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f7798444c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f779846df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f77979f900 con 0x55f779844a00 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f779745ef0 con 0x55f779839820 2024-04-26T01:58:39.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd998d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f77979b8c0 con 0x55f77983a140 2024-04-26T01:58:39.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f779839c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f779839c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:49660/0 (socket says 172.21.15.37:49660) 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdd4a00640 1 -- 172.21.15.37:0/3689654067 learned_addr learned my addr 172.21.15.37:0/3689654067 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.159+0000 7fbdcf400640 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f779846df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdd4a00640 1 -- 172.21.15.37:0/3689654067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 msgr2=0x55f7798444c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f7798444c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdd4a00640 1 -- 172.21.15.37:0/3689654067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 msgr2=0x55f779846df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f779846df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdd4a00640 1 -- 172.21.15.37:0/3689654067 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7794b2210 con 0x55f779839820 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f779839c00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011790 tx=0x7fbdc8011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd3139fb20e9e894 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.160+0000 7fbdc5200640 1 -- 172.21.15.37:0/3689654067 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc8003c30 con 0x55f779839820 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdc5200640 1 -- 172.21.15.37:0/3689654067 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc800fe50 con 0x55f779839820 2024-04-26T01:58:39.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdc5200640 1 -- 172.21.15.37:0/3689654067 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc80193e0 con 0x55f779839820 2024-04-26T01:58:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdccc00640 1 -- 172.21.15.37:0/2972191597 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fbdc00039f0 con 0x7fbdd01021b0 2024-04-26T01:58:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdd998d740 1 -- 172.21.15.37:0/3689654067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 msgr2=0x55f779839c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdd998d740 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f779839c00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011790 tx=0x7fbdc8011c50 comp rx=0 tx=0).stop 2024-04-26T01:58:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdd998d740 1 -- 172.21.15.37:0/3689654067 shutdown_connections 2024-04-26T01:58:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdd998d740 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f779846df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdd998d740 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f7798444c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.161+0000 7fbdd998d740 1 --2- 172.21.15.37:0/3689654067 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f779839c00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.162+0000 7fbdd998d740 1 -- 172.21.15.37:0/3689654067 >> 172.21.15.37:0/3689654067 conn(0x55f779835d30 msgr2=0x55f779836110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.162+0000 7fbdd998d740 1 -- 172.21.15.37:0/3689654067 shutdown_connections 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.162+0000 7fbdd998d740 1 -- 172.21.15.37:0/3689654067 wait complete. 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.162+0000 7fbdd998d740 1 -- start start 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 Processor -- start 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdccc00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fbdb40a3d40 0x7fbdb40a6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdccc00640 1 -- 172.21.15.37:0/2972191597 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbdc00f0570 con 0x7fbdd01021b0 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fbdb40a3d40 0x7fbdb40a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 0x55f7798419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f77983a140 0x55f779841f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f779844a00 0x55f779842460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f779847630 con 0x55f779839820 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55f779759310 con 0x55f77983a140 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55f7798477b0 con 0x55f779844a00 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.163+0000 7fbdd998d740 10 client.0 _ll_register_callbacks cb 0x55f779727cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 0x55f7798419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 0x55f7798419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44856/0 (socket says 172.21.15.37:44856) 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 -- 172.21.15.37:0/624179361 learned_addr learned my addr 172.21.15.37:0/624179361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f779844a00 0x55f779842460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 -- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f779844a00 msgr2=0x55f779842460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f779844a00 0x55f779842460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 -- 172.21.15.37:0/624179361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f77983a140 msgr2=0x55f779841f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f77983a140 0x55f779841f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 -- 172.21.15.37:0/624179361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77957e620 con 0x55f779839820 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fbdb40a3d40 0x7fbdb40a6200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011760 tx=0x7fbdc8005bd0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 0x55f7798419e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc00bbfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f5a9560ead60d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.164+0000 7fbdb9600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdb9600640 1 -- 172.21.15.37:0/624179361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0030450 con 0x55f779839820 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc800fe50 con 0x55f779839820 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdb9600640 1 -- 172.21.15.37:0/624179361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbdc00eddf0 con 0x55f779839820 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdb9600640 1 -- 172.21.15.37:0/624179361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbdc00da050 con 0x55f779839820 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdd998d740 10 client.15048 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f7798538c0 con 0x55f779839820 2024-04-26T01:58:39.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.165+0000 7fbdb9600640 1 -- 172.21.15.37:0/624179361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc800fe50 con 0x55f779839820 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdb9600640 1 -- 172.21.15.37:0/624179361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fbdc00f0070 con 0x55f779839820 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdd998d740 10 client.15048 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdd998d740 10 client.15048 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdd998d740 20 client.15048 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdd998d740 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f779854d50 0x55f779857180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdd998d740 4 client.15048 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7797b38b0 con 0x55f779854d50 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.166+0000 7fbdcf400640 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f779854d50 0x55f779857180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.167+0000 7fbdcf400640 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f779854d50 0x55f779857180 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447bc93d23c3af7a server_cookie=ea4e01d2a57f00d2 in_seq=0 out_seq=0 2024-04-26T01:58:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.167+0000 7fbdb9600640 10 client.15048 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.167+0000 7fbdb9600640 1 -- 172.21.15.37:0/624179361 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f7797b38b0 con 0x55f779854d50 2024-04-26T01:58:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.167+0000 7fbdb9600640 10 client.15048 handle_command_reply: tid=0 2024-04-26T01:58:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 client.15048 shutdown 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 msgr2=0x55f7798419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 0x55f7798419e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc00bbfa0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 shutdown_connections 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f779854d50 0x55f779857180 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f779844a00 0x55f779842460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f77983a140 0x55f779841f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 --2- 172.21.15.37:0/624179361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779839820 0x55f7798419e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 >> 172.21.15.37:0/624179361 conn(0x55f779835d30 msgr2=0x55f77983ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdb9600640 0 client.15048 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 shutdown_connections 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 1 -- 172.21.15.37:0/624179361 wait complete. 2024-04-26T01:58:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.168+0000 7fbdd998d740 20 client.15048 trim_cache size 0 max 16384 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 Processor -- start 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 -- start start 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f7798504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77983c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f7798420b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01add30 con 0x55f779844a00 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01b05b0 con 0x55f779839820 2024-04-26T01:58:39.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd998d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01ae3a0 con 0x55f77983a140 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdd4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f7798420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f7798504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77983c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f7798504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:49668/0 (socket says 172.21.15.37:49668) 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.185+0000 7fbdcfe00640 1 -- 172.21.15.37:0/758117079 learned_addr learned my addr 172.21.15.37:0/758117079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdcfe00640 1 -- 172.21.15.37:0/758117079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 msgr2=0x55f77983c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77983c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdcfe00640 1 -- 172.21.15.37:0/758117079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 msgr2=0x55f7798420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f7798420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdcfe00640 1 -- 172.21.15.37:0/758117079 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f779847160 con 0x55f779839820 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f7798504f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00b8d70 tx=0x7fbdc00bc010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=351eff5b37a25cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdbbe00640 1 -- 172.21.15.37:0/758117079 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0030900 con 0x55f779839820 2024-04-26T01:58:39.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdbbe00640 1 -- 172.21.15.37:0/758117079 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00db240 con 0x55f779839820 2024-04-26T01:58:39.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.186+0000 7fbdbbe00640 1 -- 172.21.15.37:0/758117079 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00c5010 con 0x55f779839820 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 -- 172.21.15.37:0/758117079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 msgr2=0x55f7798504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f7798504f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbdc00b8d70 tx=0x7fbdc00bc010 comp rx=0 tx=0).stop 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 -- 172.21.15.37:0/758117079 shutdown_connections 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f7798420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77983c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 --2- 172.21.15.37:0/758117079 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f7798504f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 -- 172.21.15.37:0/758117079 >> 172.21.15.37:0/758117079 conn(0x55f779835d30 msgr2=0x55f7798419a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 -- 172.21.15.37:0/758117079 shutdown_connections 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.187+0000 7fbdd998d740 1 -- 172.21.15.37:0/758117079 wait complete. 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.188+0000 7fbdd998d740 1 -- start start 2024-04-26T01:58:39.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.188+0000 7fbdd998d740 1 Processor -- start 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f77984cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77984a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f77984a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01aea70 con 0x55f779844a00 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01afee0 con 0x55f779839820 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd01af140 con 0x55f77983a140 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd998d740 10 client.0 _ll_register_callbacks cb 0x7fbdc00bc170 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f77984a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.191 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f77984a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44870/0 (socket says 172.21.15.37:44870) 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdcf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77984a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdd4a00640 1 -- 172.21.15.37:0/2492966420 learned_addr learned my addr 172.21.15.37:0/2492966420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.189+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f77984cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 msgr2=0x55f77984a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77984a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2492966420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 msgr2=0x55f77984a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f77984a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdcfe00640 1 -- 172.21.15.37:0/2492966420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77984abf0 con 0x55f779839820 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdd4a00640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f77984a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdcfe00640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f77984cc80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0005b60 tx=0x7fbdc00bc2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb78e559ae527992 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdb9600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdb9600640 1 -- 172.21.15.37:0/2492966420 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0030900 con 0x55f779839820 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdb9600640 1 -- 172.21.15.37:0/2492966420 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fbdc00d5d30 con 0x55f779839820 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.190+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc00db240 con 0x55f779839820 2024-04-26T01:58:39.192 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdb9600640 1 -- 172.21.15.37:0/2492966420 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fbdc0030450 con 0x55f779839820 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 10 client.25145 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55f77984abf0 con 0x55f779839820 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdb9600640 1 -- 172.21.15.37:0/2492966420 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc00db240 con 0x55f779839820 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdb9600640 1 -- 172.21.15.37:0/2492966420 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fbdc00bb260 con 0x55f779839820 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 10 client.25145 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 10 client.25145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 20 client.25145 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f7798bb4a0 0x55f7798bd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 4 client.25145 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.191+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f7797ad320 con 0x55f7798bb4a0 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.192+0000 7fbdcf400640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f7798bb4a0 0x55f7798bd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.192+0000 7fbdcf400640 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f7798bb4a0 0x55f7798bd890 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a3caf4b4362a22 server_cookie=c7cf696164e4b821 in_seq=0 out_seq=0 2024-04-26T01:58:39.193 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.192+0000 7fbdb9600640 10 client.25145 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.201+0000 7fbdb9600640 1 -- 172.21.15.37:0/2492966420 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f7797ad320 con 0x55f7798bb4a0 2024-04-26T01:58:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.201+0000 7fbdb9600640 10 client.25145 handle_command_reply: tid=0 2024-04-26T01:58:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.201+0000 7fbdd998d740 1 client.25145 shutdown 2024-04-26T01:58:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 msgr2=0x55f77984cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f77984cc80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0005b60 tx=0x7fbdc00bc2a0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 shutdown_connections 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55f7798bb4a0 0x55f7798bd890 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f779844a00 0x55f77984a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55f77983a140 0x55f77984a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdb9600640 0 client.25145 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 --2- 172.21.15.37:0/2492966420 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55f779839820 0x55f77984cc80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 >> 172.21.15.37:0/2492966420 conn(0x55f779835d30 msgr2=0x55f77983fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 shutdown_connections 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 1 -- 172.21.15.37:0/2492966420 wait complete. 2024-04-26T01:58:39.202 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.202+0000 7fbdd998d740 20 client.25145 trim_cache size 0 max 16384 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fbdb40a3d40 msgr2=0x7fbdb40a6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fbdb40a3d40 0x7fbdb40a6200 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbdc8011760 tx=0x7fbdc8005bd0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 msgr2=0x7fbdd01a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd01a0610 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0035040 tx=0x7fbdc0032d50 comp rx=0 tx=0).stop 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 shutdown_connections 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fbdb40a3d40 0x7fbdb40a6200 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbdd019a780 0x7fbdd019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fbdd0107280 0x7fbdd01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 --2- 172.21.15.37:0/2972191597 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fbdd01021b0 0x7fbdd01a0610 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 >> 172.21.15.37:0/2972191597 conn(0x7fbdd00fda10 msgr2=0x7fbdd00fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 shutdown_connections 2024-04-26T01:58:39.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:39.203+0000 7fbdd6a00640 1 -- 172.21.15.37:0/2972191597 wait complete. 2024-04-26T01:58:40.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:39 smithi071 ceph-mon[32578]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-26T01:58:40.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:39 smithi037 ceph-mon[27906]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-26T01:58:40.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:39 smithi045 ceph-mon[32520]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-04-26T01:58:42.057 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:41 smithi045 ceph-mon[32520]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-26T01:58:42.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:41 smithi071 ceph-mon[32578]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-26T01:58:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:41 smithi037 ceph-mon[27906]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 767 B/s wr, 1 op/s 2024-04-26T01:58:44.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:43 smithi071 ceph-mon[32578]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-26T01:58:44.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:43 smithi037 ceph-mon[27906]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-26T01:58:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:43 smithi045 ceph-mon[32520]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-04-26T01:58:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:45 smithi071 ceph-mon[32578]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:58:46.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:45 smithi037 ceph-mon[27906]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:58:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:45 smithi045 ceph-mon[32520]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:58:47.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:47 smithi037 ceph-mon[27906]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:58:48.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:47 smithi071 ceph-mon[32578]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:58:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:47 smithi045 ceph-mon[32520]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:58:50.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:49 smithi071 ceph-mon[32578]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:58:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:49 smithi037 ceph-mon[27906]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:58:50.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:49 smithi045 ceph-mon[32520]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:58:51.614 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.6 2024-04-26T01:58:51.614 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:58:51.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:51 smithi037 ceph-mon[27906]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:58:52.040 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:52.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:51 smithi071 ceph-mon[32578]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:58:52.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:51 smithi045 ceph-mon[32520]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 -- 172.21.15.37:0/2929927052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4624074c20 msgr2=0x7f462410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 --2- 172.21.15.37:0/2929927052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4624074c20 0x7f462410e2c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f460c009860 tx=0x7f460c02a3d0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 -- 172.21.15.37:0/2929927052 shutdown_connections 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 --2- 172.21.15.37:0/2929927052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f462410e800 0x7f4624110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 --2- 172.21.15.37:0/2929927052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4624074c20 0x7f462410e2c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 --2- 172.21.15.37:0/2929927052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4624074300 0x7f46240746e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 -- 172.21.15.37:0/2929927052 >> 172.21.15.37:0/2929927052 conn(0x7f46240fda10 msgr2=0x7f46240ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.336+0000 7f462a000640 1 -- 172.21.15.37:0/2929927052 shutdown_connections 2024-04-26T01:58:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.337+0000 7f462a000640 1 -- 172.21.15.37:0/2929927052 wait complete. 2024-04-26T01:58:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.338+0000 7f462a000640 1 Processor -- start 2024-04-26T01:58:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.338+0000 7f462a000640 1 -- start start 2024-04-26T01:58:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f462a000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 0x7f46241a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f462a000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4624074c20 0x7f46241a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f462a000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462410e800 0x7f462419a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f462a000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4624114470 con 0x7f4624074300 2024-04-26T01:58:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f462a000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f46241142f0 con 0x7f4624074c20 2024-04-26T01:58:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f462a000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f46241145f0 con 0x7f462410e800 2024-04-26T01:58:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f4623400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 0x7f46241a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f4623400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 0x7f46241a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43314/0 (socket says 172.21.15.37:43314) 2024-04-26T01:58:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f4623400640 1 -- 172.21.15.37:0/3401292870 learned_addr learned my addr 172.21.15.37:0/3401292870 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f4622a00640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4624074c20 0x7f46241a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.339+0000 7f4623e00640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462410e800 0x7f462419a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623400640 1 -- 172.21.15.37:0/3401292870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462410e800 msgr2=0x7f462419a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623400640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462410e800 0x7f462419a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623400640 1 -- 172.21.15.37:0/3401292870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4624074c20 msgr2=0x7f46241a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623400640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4624074c20 0x7f46241a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623400640 1 -- 172.21.15.37:0/3401292870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462419b000 con 0x7f4624074300 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623e00640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462410e800 0x7f462419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.340+0000 7f4623400640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 0x7f46241a06c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f461c00ba50 tx=0x7f461c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.341+0000 7f461be00640 1 -- 172.21.15.37:0/3401292870 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c019070 con 0x7f4624074300 2024-04-26T01:58:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.341+0000 7f461be00640 1 -- 172.21.15.37:0/3401292870 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c00cbe0 con 0x7f4624074300 2024-04-26T01:58:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.341+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462419b2f0 con 0x7f4624074300 2024-04-26T01:58:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.341+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a7510 con 0x7f4624074300 2024-04-26T01:58:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.341+0000 7f461be00640 1 -- 172.21.15.37:0/3401292870 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0075d0 con 0x7f4624074300 2024-04-26T01:58:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.346+0000 7f461be00640 1 -- 172.21.15.37:0/3401292870 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f461c01c070 con 0x7f4624074300 2024-04-26T01:58:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.346+0000 7f462ce60740 1 Processor -- start 2024-04-26T01:58:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.346+0000 7f462ce60740 1 -- start start 2024-04-26T01:58:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.347+0000 7f461be00640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f46080a3e30 0x7f46080a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.347+0000 7f461be00640 1 -- 172.21.15.37:0/3401292870 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f461c0cfec0 con 0x7f4624074300 2024-04-26T01:58:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.347+0000 7f462ce60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3496410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f462ce60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e34a0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.347+0000 7f4623e00640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f46080a3e30 0x7f46080a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f4623400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3496410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f462ce60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e34a1210 0x5566e34a3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f462ce60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566e33f3900 con 0x5566e34a1210 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f462ce60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566e3399ef0 con 0x5566e3496950 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f462ce60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566e33ef8c0 con 0x5566e3496030 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f4623400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3496410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45986/0 (socket says 172.21.15.37:45986) 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f4623400640 1 -- 172.21.15.37:0/819979309 learned_addr learned my addr 172.21.15.37:0/819979309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.348+0000 7f4622a00640 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e34a0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623400640 1 -- 172.21.15.37:0/819979309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 msgr2=0x5566e34a0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623400640 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e34a0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623400640 1 -- 172.21.15.37:0/819979309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e34a1210 msgr2=0x5566e34a3600 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:58:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623400640 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e34a1210 0x5566e34a3600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623400640 1 -- 172.21.15.37:0/819979309 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e3106210 con 0x5566e3496030 2024-04-26T01:58:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623400640 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3496410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f461c098770 tx=0x7f461c09bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e68c79fa0709eee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:52.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4623e00640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f46080a3e30 0x7f46080a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f461400bc30 tx=0x7f4614005f10 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.349+0000 7f4613e00640 1 -- 172.21.15.37:0/819979309 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0b5070 con 0x5566e3496030 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.350+0000 7f4613e00640 1 -- 172.21.15.37:0/819979309 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c099e40 con 0x5566e3496030 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.350+0000 7f4613e00640 1 -- 172.21.15.37:0/819979309 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c0cd740 con 0x5566e3496030 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.350+0000 7f462ce60740 1 -- 172.21.15.37:0/819979309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 msgr2=0x5566e3496410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.350+0000 7f462ce60740 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3496410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f461c098770 tx=0x7f461c09bda0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 -- 172.21.15.37:0/819979309 shutdown_connections 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e34a1210 0x5566e34a3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e34a0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 --2- 172.21.15.37:0/819979309 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3496410 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 -- 172.21.15.37:0/819979309 >> 172.21.15.37:0/819979309 conn(0x5566e3492540 msgr2=0x5566e3492920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 -- 172.21.15.37:0/819979309 shutdown_connections 2024-04-26T01:58:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 -- 172.21.15.37:0/819979309 wait complete. 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.351+0000 7f462ce60740 1 -- start start 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 Processor -- start 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e349e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e34a1210 0x5566e349ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5566e34a3e40 con 0x5566e3496030 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5566e33ad310 con 0x5566e3496950 2024-04-26T01:58:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5566e34a3fc0 con 0x5566e34a1210 2024-04-26T01:58:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.352+0000 7f462ce60740 10 client.0 _ll_register_callbacks cb 0x5566e337bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e34a1210 0x5566e349ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43322/0 (socket says 172.21.15.37:43322) 2024-04-26T01:58:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 -- 172.21.15.37:0/364552129 learned_addr learned my addr 172.21.15.37:0/364552129 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4622a00640 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e349e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 -- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e34a1210 msgr2=0x5566e349ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e34a1210 0x5566e349ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 -- 172.21.15.37:0/364552129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 msgr2=0x5566e349e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e349e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.353+0000 7f4623400640 1 -- 172.21.15.37:0/364552129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e31d2620 con 0x5566e3496030 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.354+0000 7f4623400640 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349e270 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f461c09baa0 tx=0x7f461c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1af5e9bfa5aa6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.354+0000 7f4610c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.354+0000 7f4610c00640 1 -- 172.21.15.37:0/364552129 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c07f060 con 0x5566e3496030 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.354+0000 7f4610c00640 1 -- 172.21.15.37:0/364552129 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f461c07adc0 con 0x5566e3496030 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.354+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c099e40 con 0x5566e3496030 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f4610c00640 1 -- 172.21.15.37:0/364552129 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f461c0b5040 con 0x5566e3496030 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 10 client.15063 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5566e34b01c0 con 0x5566e3496030 2024-04-26T01:58:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f4610c00640 1 -- 172.21.15.37:0/364552129 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c099e40 con 0x5566e3496030 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f4610c00640 1 -- 172.21.15.37:0/364552129 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f461c0cd740 con 0x5566e3496030 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 10 client.15063 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 10 client.15063 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 20 client.15063 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e34b1510 0x5566e34b3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 4 client.15063 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.355+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566e34078b0 con 0x5566e34b1510 2024-04-26T01:58:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.356+0000 7f4622a00640 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e34b1510 0x5566e34b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.356+0000 7f4622a00640 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e34b1510 0x5566e34b3900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e5081de95e775d server_cookie=8cbe176374ed092c in_seq=0 out_seq=0 2024-04-26T01:58:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.356+0000 7f4610c00640 10 client.15063 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.357+0000 7f4610c00640 1 -- 172.21.15.37:0/364552129 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566e34078b0 con 0x5566e34b1510 2024-04-26T01:58:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.357+0000 7f4610c00640 10 client.15063 handle_command_reply: tid=0 2024-04-26T01:58:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.359+0000 7f462ce60740 1 client.15063 shutdown 2024-04-26T01:58:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 msgr2=0x5566e349e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349e270 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f461c09baa0 tx=0x7f461c009a20 comp rx=0 tx=0).stop 2024-04-26T01:58:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 shutdown_connections 2024-04-26T01:58:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e34b1510 0x5566e34b3900 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e34a1210 0x5566e349ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e3496950 0x5566e349e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 --2- 172.21.15.37:0/364552129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349e270 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 >> 172.21.15.37:0/364552129 conn(0x5566e3492540 msgr2=0x5566e3488e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.361+0000 7f4610c00640 0 client.15063 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.362+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 shutdown_connections 2024-04-26T01:58:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.362+0000 7f462ce60740 1 -- 172.21.15.37:0/364552129 wait complete. 2024-04-26T01:58:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.362+0000 7f462ce60740 20 client.15063 trim_cache size 0 max 16384 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.379+0000 7f462ce60740 1 Processor -- start 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 -- start start 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496950 0x5566e3488600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e3488b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f460000c640 con 0x5566e3496030 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f460000eec0 con 0x5566e34a1210 2024-04-26T01:58:52.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f462ce60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f460000ccb0 con 0x5566e3496950 2024-04-26T01:58:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f4623400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f4623400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43334/0 (socket says 172.21.15.37:43334) 2024-04-26T01:58:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f4623400640 1 -- 172.21.15.37:0/2864479255 learned_addr learned my addr 172.21.15.37:0/2864479255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:52.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.380+0000 7f4622a00640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496950 0x5566e3488600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623e00640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e3488b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623400640 1 -- 172.21.15.37:0/2864479255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496950 msgr2=0x5566e3488600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623400640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496950 0x5566e3488600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623400640 1 -- 172.21.15.37:0/2864479255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 msgr2=0x5566e3488b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623400640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e3488b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623400640 1 -- 172.21.15.37:0/2864479255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566e349d240 con 0x5566e3496030 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4622a00640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496950 0x5566e3488600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:52.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.381+0000 7f4623e00640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e3488b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.382+0000 7f4623400640 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349cb60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f461c004a80 tx=0x7f461c09bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27be977e049df07d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.382+0000 7f4613400640 1 -- 172.21.15.37:0/2864479255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c07f050 con 0x5566e3496030 2024-04-26T01:58:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.382+0000 7f4613400640 1 -- 172.21.15.37:0/2864479255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c097e30 con 0x5566e3496030 2024-04-26T01:58:52.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.382+0000 7f4613400640 1 -- 172.21.15.37:0/2864479255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c009040 con 0x5566e3496030 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 -- 172.21.15.37:0/2864479255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 msgr2=0x5566e349cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349cb60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f461c004a80 tx=0x7f461c09bda0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 -- 172.21.15.37:0/2864479255 shutdown_connections 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e3488b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496950 0x5566e3488600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 --2- 172.21.15.37:0/2864479255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496030 0x5566e349cb60 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.383+0000 7f462ce60740 1 -- 172.21.15.37:0/2864479255 >> 172.21.15.37:0/2864479255 conn(0x5566e3492540 msgr2=0x5566e34acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:52.383 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.384+0000 7f462ce60740 1 -- 172.21.15.37:0/2864479255 shutdown_connections 2024-04-26T01:58:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.384+0000 7f462ce60740 1 -- 172.21.15.37:0/2864479255 wait complete. 2024-04-26T01:58:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.384+0000 7f462ce60740 1 -- start start 2024-04-26T01:58:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.384+0000 7f462ce60740 1 Processor -- start 2024-04-26T01:58:52.384 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3510100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 0x5566e3510640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e34b5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f460000d380 con 0x5566e3496950 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f460000e7f0 con 0x5566e34a1210 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f460000da50 con 0x5566e3496030 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.385+0000 7f462ce60740 10 client.0 _ll_register_callbacks cb 0x7f460c032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:52.385 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4622a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 0x5566e3510640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3510100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4622a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 0x5566e3510640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43344/0 (socket says 172.21.15.37:43344) 2024-04-26T01:58:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4622a00640 1 -- 172.21.15.37:0/2900007386 learned_addr learned my addr 172.21.15.37:0/2900007386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623e00640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e34b5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623400640 1 -- 172.21.15.37:0/2900007386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 msgr2=0x5566e34b5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623400640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e34b5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623400640 1 -- 172.21.15.37:0/2900007386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 msgr2=0x5566e3510640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623400640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 0x5566e3510640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4623400640 1 -- 172.21.15.37:0/2900007386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566e34b65a0 con 0x5566e3496030 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.386+0000 7f4622a00640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 0x5566e3510640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:58:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.387+0000 7f4623400640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3510100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f461c09baa0 tx=0x7f461c0cd4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec03f2d9be6d4399 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.387+0000 7f4610c00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:58:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.387+0000 7f4610c00640 1 -- 172.21.15.37:0/2900007386 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c07f070 con 0x5566e3496030 2024-04-26T01:58:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.387+0000 7f4610c00640 1 -- 172.21.15.37:0/2900007386 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f461c0cd6b0 con 0x5566e3496030 2024-04-26T01:58:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.387+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c097e30 con 0x5566e3496030 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.388+0000 7f4610c00640 1 -- 172.21.15.37:0/2900007386 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f461c09bb70 con 0x5566e3496030 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.388+0000 7f462ce60740 10 client.25204 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.388+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5566e34b65a0 con 0x5566e3496030 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.388+0000 7f4610c00640 1 -- 172.21.15.37:0/2900007386 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461c097e30 con 0x5566e3496030 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.388+0000 7f4610c00640 1 -- 172.21.15.37:0/2900007386 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f461c07ac30 con 0x5566e3496030 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f462ce60740 10 client.25204 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f462ce60740 10 client.25204 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f462ce60740 20 client.25204 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f462ce60740 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e3517a30 0x5566e3519e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f462ce60740 4 client.25204 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566e3401320 con 0x5566e3517a30 2024-04-26T01:58:52.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.389+0000 7f4622a00640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e3517a30 0x5566e3519e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.391+0000 7f4622a00640 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e3517a30 0x5566e3519e20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5845ec114db3a6 server_cookie=89d491ac1dff78a4 in_seq=0 out_seq=0 2024-04-26T01:58:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.391+0000 7f4610c00640 10 client.25204 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.395+0000 7f4610c00640 1 -- 172.21.15.37:0/2900007386 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566e3401320 con 0x5566e3517a30 2024-04-26T01:58:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.395+0000 7f4610c00640 10 client.25204 handle_command_reply: tid=0 2024-04-26T01:58:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.396+0000 7f462ce60740 1 client.25204 shutdown 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 msgr2=0x5566e3510100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3510100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f461c09baa0 tx=0x7f461c0cd4b0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 shutdown_connections 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5566e3517a30 0x5566e3519e20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5566e34a1210 0x5566e34b5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5566e3496950 0x5566e3510640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 --2- 172.21.15.37:0/2900007386 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5566e3496030 0x5566e3510100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f4610c00640 0 client.25204 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.397+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 >> 172.21.15.37:0/2900007386 conn(0x5566e3492540 msgr2=0x5566e34acaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.398+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 shutdown_connections 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.398+0000 7f462ce60740 1 -- 172.21.15.37:0/2900007386 wait complete. 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.398+0000 7f462ce60740 20 client.25204 trim_cache size 0 max 16384 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:58:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f46080a3e30 msgr2=0x7f46080a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f46080a3e30 0x7f46080a62f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f461400bc30 tx=0x7f4614005f10 comp rx=0 tx=0).stop 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 msgr2=0x7f46241a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 0x7f46241a06c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f461c00ba50 tx=0x7f461c00bf10 comp rx=0 tx=0).stop 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 shutdown_connections 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f46080a3e30 0x7f46080a62f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f462410e800 0x7f462419a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4624074c20 0x7f46241a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 --2- 172.21.15.37:0/3401292870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4624074300 0x7f46241a06c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.400+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 >> 172.21.15.37:0/3401292870 conn(0x7f46240fda10 msgr2=0x7f46240ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.403+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 shutdown_connections 2024-04-26T01:58:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:58:52.403+0000 7f462a000640 1 -- 172.21.15.37:0/3401292870 wait complete. 2024-04-26T01:58:54.079 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:53 smithi037 ceph-mon[27906]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-26T01:58:54.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:53 smithi071 ceph-mon[32578]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-26T01:58:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:53 smithi045 ceph-mon[32520]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 -- 172.21.15.37:0/144839644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806a750 msgr2=0x7f45a806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 --2- 172.21.15.37:0/144839644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806a750 0x7f45a806abb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f459400faf0 tx=0x7f459400fe00 comp rx=0 tx=0).stop 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 -- 172.21.15.37:0/144839644 shutdown_connections 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 --2- 172.21.15.37:0/144839644 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45a806b0f0 0x7f45a810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 --2- 172.21.15.37:0/144839644 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a8108a30 0x7f45a8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 --2- 172.21.15.37:0/144839644 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806a750 0x7f45a806abb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 -- 172.21.15.37:0/144839644 >> 172.21.15.37:0/144839644 conn(0x7f45a80788f0 msgr2=0x7f45a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 -- 172.21.15.37:0/144839644 shutdown_connections 2024-04-26T01:58:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.445+0000 7f45ade00700 1 -- 172.21.15.37:0/144839644 wait complete. 2024-04-26T01:58:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.446+0000 7f45ade00700 1 Processor -- start 2024-04-26T01:58:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 -- start start 2024-04-26T01:58:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 0x7f45a819dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806b0f0 0x7f45a8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45a8108a30 0x7f45a8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8110ac0 con 0x7f45a806b0f0 2024-04-26T01:58:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8110940 con 0x7f45a8108a30 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45ade00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f45a8110c40 con 0x7f45a806a750 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.447+0000 7f45a6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 0x7f45a819dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 0x7f45a819dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46034/0 (socket says 172.21.15.37:46034) 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 -- 172.21.15.37:0/1006901052 learned_addr learned my addr 172.21.15.37:0/1006901052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6000700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806b0f0 0x7f45a8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a7400700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45a8108a30 0x7f45a8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 -- 172.21.15.37:0/1006901052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45a8108a30 msgr2=0x7f45a8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45a8108a30 0x7f45a8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 -- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806b0f0 msgr2=0x7f45a8196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806b0f0 0x7f45a8196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.448+0000 7f45a6a00700 1 -- 172.21.15.37:0/1006901052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a81979b0 con 0x7f45a806a750 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.449+0000 7f45a6a00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 0x7f45a819dd70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f459800d950 tx=0x7f459800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.449+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4598016070 con 0x7f45a806a750 2024-04-26T01:58:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.449+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f459800fd40 con 0x7f45a806a750 2024-04-26T01:58:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.449+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459800e3b0 con 0x7f45a806a750 2024-04-26T01:58:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.449+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45a819b8a0 con 0x7f45a806a750 2024-04-26T01:58:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.449+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45a819bd60 con 0x7f45a806a750 2024-04-26T01:58:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.451+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45a8197c40 con 0x7f45a806a750 2024-04-26T01:58:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.452+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f4598019070 con 0x7f45a806a750 2024-04-26T01:58:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.454+0000 7f45aca00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f458c0a3ed0 0x7f458c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.454+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f45980d0520 con 0x7f45a806a750 2024-04-26T01:58:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.455+0000 7f45a6000700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f458c0a3ed0 0x7f458c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.455+0000 7f45a6000700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f458c0a3ed0 0x7f458c0a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4594009870 tx=0x7f4594005fb0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.456+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f459809c940 con 0x7f45a806a750 2024-04-26T01:58:54.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.730+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45a8198330 con 0x7f45a806a750 2024-04-26T01:58:54.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.731+0000 7f45aca00700 1 -- 172.21.15.37:0/1006901052 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f459809c240 con 0x7f45a806a750 2024-04-26T01:58:54.731 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:58:54.731 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:58:54.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f458c0a3ed0 msgr2=0x7f458c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:54.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f458c0a3ed0 0x7f458c0a6390 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4594009870 tx=0x7f4594005fb0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 msgr2=0x7f45a819dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 0x7f45a819dd70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f459800d950 tx=0x7f459800dc60 comp rx=0 tx=0).stop 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 shutdown_connections 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f458c0a3ed0 0x7f458c0a6390 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f45a8108a30 0x7f45a8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f45a806a750 0x7f45a819dd70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 --2- 172.21.15.37:0/1006901052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45a806b0f0 0x7f45a8196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:54.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.734+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 >> 172.21.15.37:0/1006901052 conn(0x7f45a80788f0 msgr2=0x7f45a80ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:54.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.735+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 shutdown_connections 2024-04-26T01:58:54.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:54.735+0000 7f45ade00700 1 -- 172.21.15.37:0/1006901052 wait complete. 2024-04-26T01:58:54.736 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:58:55.121 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:54 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1006901052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:55.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:54 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1006901052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:55.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:54 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1006901052' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:58:55.604 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0' 2024-04-26T01:58:55.605 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db / --timeout 60 --expiration 85.6 --await-for 42.0 2024-04-26T01:58:55.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:55 smithi037 ceph-mon[27906]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 682 B/s wr, 3 op/s 2024-04-26T01:58:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:58:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:55 smithi071 ceph-mon[32578]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 682 B/s wr, 3 op/s 2024-04-26T01:58:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:55 smithi045 ceph-mon[32520]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 682 B/s wr, 3 op/s 2024-04-26T01:58:57.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:57 smithi037 ceph-mon[27906]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T01:58:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:57 smithi071 ceph-mon[32578]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T01:58:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:57 smithi045 ceph-mon[32520]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T01:58:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 -- 172.21.15.37:0/1546803859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc1043b0 msgr2=0x7f6efc104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 --2- 172.21.15.37:0/1546803859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc1043b0 0x7f6efc104830 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8009990 tx=0x7f6ee8009ca0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 -- 172.21.15.37:0/1546803859 shutdown_connections 2024-04-26T01:58:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 --2- 172.21.15.37:0/1546803859 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6efc104d70 0x7f6efc10f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 --2- 172.21.15.37:0/1546803859 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 --2- 172.21.15.37:0/1546803859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc1043b0 0x7f6efc104830 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 -- 172.21.15.37:0/1546803859 >> 172.21.15.37:0/1546803859 conn(0x7f6efc0fe3a0 msgr2=0x7f6efc1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 -- 172.21.15.37:0/1546803859 shutdown_connections 2024-04-26T01:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.493+0000 7f6f01000700 1 -- 172.21.15.37:0/1546803859 wait complete. 2024-04-26T01:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.494+0000 7f6f01000700 1 Processor -- start 2024-04-26T01:58:58.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 -- start start 2024-04-26T01:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc19b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6efc1043b0 0x7f6efc19b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc104d70 0x7f6efc19fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112bc0 con 0x7f6efc104d70 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112a40 con 0x7f6efc102c60 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.495+0000 7f6f01000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc112d40 con 0x7f6efc1043b0 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc19b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc104d70 0x7f6efc19fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efbe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc104d70 0x7f6efc19fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43380/0 (socket says 172.21.15.37:43380) 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc19b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:58580/0 (socket says 172.21.15.37:58580) 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efbe00700 1 -- 172.21.15.37:0/970164073 learned_addr learned my addr 172.21.15.37:0/970164073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efaa00700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6efc1043b0 0x7f6efc19b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 -- 172.21.15.37:0/970164073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6efc1043b0 msgr2=0x7f6efc19b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6efc1043b0 0x7f6efc19b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 -- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc104d70 msgr2=0x7f6efc19fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc104d70 0x7f6efc19fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.496+0000 7f6efb400700 1 -- 172.21.15.37:0/970164073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6efc1a01a0 con 0x7f6efc102c60 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.497+0000 7f6efb400700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc19b190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c3d0 tx=0x7f6eec00c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.497+0000 7f6ef3e00700 1 -- 172.21.15.37:0/970164073 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec023070 con 0x7f6efc102c60 2024-04-26T01:58:58.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.497+0000 7f6ef3e00700 1 -- 172.21.15.37:0/970164073 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6eec00ce20 con 0x7f6efc102c60 2024-04-26T01:58:58.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.497+0000 7f6ef3e00700 1 -- 172.21.15.37:0/970164073 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec016510 con 0x7f6efc102c60 2024-04-26T01:58:58.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.497+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6efc1a0490 con 0x7f6efc102c60 2024-04-26T01:58:58.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.497+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6efc1097c0 con 0x7f6efc102c60 2024-04-26T01:58:58.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.503+0000 7f6ef3e00700 1 -- 172.21.15.37:0/970164073 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6eec026070 con 0x7f6efc102c60 2024-04-26T01:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.504+0000 7f6f0cf49b80 1 Processor -- start 2024-04-26T01:58:58.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.504+0000 7f6f0cf49b80 1 -- start start 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6ef3e00700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6ee00a4070 0x7f6ee00a6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e61249b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 0x55e6124a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6ef3e00700 1 -- 172.21.15.37:0/970164073 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6eec0d88c0 con 0x7f6efc102c60 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6124a6700 0x55e6124a8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6123ee020 con 0x55e6124a6700 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e6123948c0 con 0x55e61249b850 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.505+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e6123e9fe0 con 0x55e61249af30 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efbe00700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6ee00a4070 0x7f6ee00a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 0x55e6124a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 0x55e6124a61c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:58588/0 (socket says 172.21.15.37:58588) 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 -- 172.21.15.37:0/4042672750 learned_addr learned my addr 172.21.15.37:0/4042672750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efb400700 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e61249b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 -- 172.21.15.37:0/4042672750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 msgr2=0x55e61249b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e61249b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 -- 172.21.15.37:0/4042672750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6124a6700 msgr2=0x55e6124a8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6124a6700 0x55e6124a8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.506+0000 7f6efaa00700 1 -- 172.21.15.37:0/4042672750 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61238a810 con 0x55e61249b850 2024-04-26T01:58:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.507+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 0x55e6124a61c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8015000 tx=0x7f6ee80096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78fe04eb75f3b2ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.507+0000 7f6ef0c00700 1 -- 172.21.15.37:0/4042672750 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8027070 con 0x55e61249b850 2024-04-26T01:58:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.507+0000 7f6ef0c00700 1 -- 172.21.15.37:0/4042672750 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ee80056d0 con 0x55e61249b850 2024-04-26T01:58:58.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.507+0000 7f6ef0c00700 1 -- 172.21.15.37:0/4042672750 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee8003bb0 con 0x55e61249b850 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.508+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4042672750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 msgr2=0x55e6124a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.508+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 0x55e6124a61c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8015000 tx=0x7f6ee80096a0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.508+0000 7f6efbe00700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6ee00a4070 0x7f6ee00a6530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009fd0 tx=0x7f6ef4009920 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4042672750 shutdown_connections 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6124a6700 0x55e6124a8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e61249b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4042672750 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e61249b850 0x55e6124a61c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4042672750 >> 172.21.15.37:0/4042672750 conn(0x55e6123ca0e0 msgr2=0x55e612497480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4042672750 shutdown_connections 2024-04-26T01:58:58.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4042672750 wait complete. 2024-04-26T01:58:58.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.509+0000 7f6f0cf49b80 1 -- start start 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.510+0000 7f6f0cf49b80 1 Processor -- start 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e6124a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124a3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6124a9130 con 0x55e61249b850 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e6123a79f0 con 0x55e6124a6700 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e6124a92b0 con 0x55e61249af30 2024-04-26T01:58:58.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6f0cf49b80 10 client.0 _ll_register_callbacks cb 0x55e6123868b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6efb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e6124a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6efb400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e6124a5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:46070/0 (socket says 172.21.15.37:46070) 2024-04-26T01:58:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.511+0000 7f6efb400700 1 -- 172.21.15.37:0/1558143584 learned_addr learned my addr 172.21.15.37:0/1558143584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efbe00700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124a3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efaa00700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efb400700 1 -- 172.21.15.37:0/1558143584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 msgr2=0x55e6124a3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efb400700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124a3660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efb400700 1 -- 172.21.15.37:0/1558143584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 msgr2=0x55e6124a3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efb400700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efb400700 1 -- 172.21.15.37:0/1558143584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61233ba60 con 0x55e61249af30 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efaa00700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6efb400700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e6124a5e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c3a0 tx=0x7f6eec01ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3bcb8dcc734328 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.512+0000 7f6ee5600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.513+0000 7f6ee5600700 1 -- 172.21.15.37:0/1558143584 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0a8990 con 0x55e61249af30 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.513+0000 7f6ee5600700 1 -- 172.21.15.37:0/1558143584 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6eec0acfd0 con 0x55e61249af30 2024-04-26T01:58:58.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.513+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee80056d0 con 0x55e61249af30 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.513+0000 7f6ee5600700 1 -- 172.21.15.37:0/1558143584 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6eec0ad250 con 0x55e61249af30 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.513+0000 7f6f0cf49b80 10 client.25222 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.513+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e612384c80 con 0x55e61249af30 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6ee5600700 1 -- 172.21.15.37:0/1558143584 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ee80056d0 con 0x55e61249af30 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6ee5600700 1 -- 172.21.15.37:0/1558143584 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f6eec0a0330 con 0x55e61249af30 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6f0cf49b80 10 client.25222 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:58.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6f0cf49b80 10 client.25222 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T01:58:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6f0cf49b80 20 client.25222 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e6124b5dc0 0x55e6124b8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6f0cf49b80 4 client.25222 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:58:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.514+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6123fb810 con 0x55e6124b5dc0 2024-04-26T01:58:58.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.515+0000 7f6efaa00700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e6124b5dc0 0x55e6124b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.515+0000 7f6efaa00700 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e6124b5dc0 0x55e6124b8240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3037fdd29787fd server_cookie=7c1c65fae866b31b in_seq=0 out_seq=0 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.515+0000 7f6ee5600700 10 client.25222 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.516+0000 7f6ee5600700 1 -- 172.21.15.37:0/1558143584 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e6123fb810 con 0x55e6124b5dc0 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.516+0000 7f6ee5600700 10 client.25222 handle_command_reply: tid=0 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.516+0000 7f6f0cf49b80 1 client.25222 shutdown 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 msgr2=0x55e6124a5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e6124a5e20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c3a0 tx=0x7f6eec01ee70 comp rx=0 tx=0).stop 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 shutdown_connections 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e6124b5dc0 0x55e6124b8240 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124a3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e61249af30 0x55e6124a5e20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1558143584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 >> 172.21.15.37:0/1558143584 conn(0x55e6123ca0e0 msgr2=0x55e612497070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6ee5600700 0 client.25222 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 shutdown_connections 2024-04-26T01:58:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1558143584 wait complete. 2024-04-26T01:58:58.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.517+0000 7f6f0cf49b80 20 client.25222 trim_cache size 0 max 16384 2024-04-26T01:58:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 Processor -- start 2024-04-26T01:58:58.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 -- start start 2024-04-26T01:58:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124bdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e612497780 0x55e6124be050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1a8f20 con 0x55e61249b850 2024-04-26T01:58:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1ab6c0 con 0x55e6124a6700 2024-04-26T01:58:58.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.558+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1a94b0 con 0x55e612497780 2024-04-26T01:58:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efb400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124bdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efaa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124bdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:58594/0 (socket says 172.21.15.37:58594) 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efaa00700 1 -- 172.21.15.37:0/1765778517 learned_addr learned my addr 172.21.15.37:0/1765778517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efbe00700 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e612497780 0x55e6124be050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efbe00700 1 -- 172.21.15.37:0/1765778517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 msgr2=0x55e6124bdb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efbe00700 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124bdb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efbe00700 1 -- 172.21.15.37:0/1765778517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 msgr2=0x55e6124a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.559+0000 7f6efbe00700 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.560+0000 7f6efbe00700 1 -- 172.21.15.37:0/1765778517 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e612496190 con 0x55e612497780 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.560+0000 7f6efbe00700 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e612497780 0x55e6124be050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d30 tx=0x7f6ef4016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=959329770ac4465e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.560+0000 7f6ee7e00700 1 -- 172.21.15.37:0/1765778517 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4029070 con 0x55e612497780 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.560+0000 7f6ee7e00700 1 -- 172.21.15.37:0/1765778517 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4017470 con 0x55e612497780 2024-04-26T01:58:58.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.560+0000 7f6ee7e00700 1 -- 172.21.15.37:0/1765778517 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4024600 con 0x55e612497780 2024-04-26T01:58:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1765778517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e612497780 msgr2=0x55e6124be050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e612497780 0x55e6124be050 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009d30 tx=0x7f6ef4016000 comp rx=0 tx=0).stop 2024-04-26T01:58:58.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1765778517 shutdown_connections 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e612497780 0x55e6124be050 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/1765778517 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e6124a6700 0x55e6124bdb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.561+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1765778517 >> 172.21.15.37:0/1765778517 conn(0x55e6123ca0e0 msgr2=0x55e6124a1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.562+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1765778517 shutdown_connections 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.562+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/1765778517 wait complete. 2024-04-26T01:58:58.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.562+0000 7f6f0cf49b80 1 -- start start 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 Processor -- start 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 0x55e6124ab030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124ab570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6124a6700 0x55e6124abab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1a9b80 con 0x55e61249b850 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1aaff0 con 0x55e612497780 2024-04-26T01:58:58.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6efc1aa250 con 0x55e6124a6700 2024-04-26T01:58:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.563+0000 7f6f0cf49b80 10 client.0 _ll_register_callbacks cb 0x55e6123868b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:58:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 0x55e6124ab030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 0x55e6124ab030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:58608/0 (socket says 172.21.15.37:58608) 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 -- 172.21.15.37:0/4190158100 learned_addr learned my addr 172.21.15.37:0/4190158100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124ab570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 -- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6124a6700 msgr2=0x55e6124abab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efbe00700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6124a6700 0x55e6124abab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6124a6700 0x55e6124abab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 -- 172.21.15.37:0/4190158100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 msgr2=0x55e6124ab570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124ab570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.564+0000 7f6efb400700 1 -- 172.21.15.37:0/4190158100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61251c440 con 0x55e612497780 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6efbe00700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6124a6700 0x55e6124abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6efb400700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 0x55e6124ab030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0a42e0 tx=0x7f6eec08de00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad5b0a4cb298de9f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:58:58.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6ee5600700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:58:58.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6ee5600700 1 -- 172.21.15.37:0/4190158100 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eec0ad250 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6ee5600700 1 -- 172.21.15.37:0/4190158100 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f6eec0acfd0 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.565+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ef4017470 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.566+0000 7f6ee5600700 1 -- 172.21.15.37:0/4190158100 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f6eec0a3c40 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.566+0000 7f6f0cf49b80 10 client.25178 fetch_fsmap learned FSMap version 19 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.566+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e61251d700 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.566+0000 7f6ee5600700 1 -- 172.21.15.37:0/4190158100 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ef4017470 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6ee5600700 1 -- 172.21.15.37:0/4190158100 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f6eec08b060 con 0x55e612497780 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6f0cf49b80 10 client.25178 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6f0cf49b80 10 client.25178 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6f0cf49b80 20 client.25178 populate_metadata read hostname 'smithi037' 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e61251dc90 0x55e612520080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6f0cf49b80 4 client.25178 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}] 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.6, "await_for": 42.0}) -- 0x55e6123f58b0 con 0x55e61251dc90 2024-04-26T01:58:58.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.567+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e61251dc90 0x55e612520080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:58:58.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.568+0000 7f6efaa00700 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e61251dc90 0x55e612520080 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6b891b383ec6520 server_cookie=de96aa8bf0d66547 in_seq=0 out_seq=0 2024-04-26T01:58:58.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.568+0000 7f6ee5600700 10 client.25178 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:58.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6ee5600700 1 -- 172.21.15.37:0/4190158100 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55e6123f58b0 con 0x55e61251dc90 2024-04-26T01:58:58.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6ee5600700 10 client.25178 handle_command_reply: tid=0 2024-04-26T01:58:58.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 client.25178 shutdown 2024-04-26T01:58:58.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 msgr2=0x55e6124ab030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 0x55e6124ab030 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0a42e0 tx=0x7f6eec08de00 comp rx=0 tx=0).stop 2024-04-26T01:58:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 shutdown_connections 2024-04-26T01:58:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e61251dc90 0x55e612520080 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e6124a6700 0x55e6124abab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e612497780 0x55e6124ab030 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 --2- 172.21.15.37:0/4190158100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e61249b850 0x55e6124ab570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 >> 172.21.15.37:0/4190158100 conn(0x55e6123ca0e0 msgr2=0x55e6124a22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6ee5600700 0 client.25178 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.578+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 shutdown_connections 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.579+0000 7f6f0cf49b80 1 -- 172.21.15.37:0/4190158100 wait complete. 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.579+0000 7f6f0cf49b80 20 client.25178 trim_cache size 0 max 16384 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T01:58:58.579 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 2, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "38b727e3": { 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 85.6, 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T01:58:58.580 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:58:58.581 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6ee00a4070 msgr2=0x7f6ee00a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6ee00a4070 0x7f6ee00a6530 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6ef4009fd0 tx=0x7f6ef4009920 comp rx=0 tx=0).stop 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 msgr2=0x7f6efc19b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc19b190 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c3d0 tx=0x7f6eec00c6e0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 shutdown_connections 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6efc104d70 0x7f6efc19fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6ee00a4070 0x7f6ee00a6530 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6efc102c60 0x7f6efc19b190 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6eec00c3d0 tx=0x7f6eec00c6e0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 --2- 172.21.15.37:0/970164073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6efc1043b0 0x7f6efc19b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 >> 172.21.15.37:0/970164073 conn(0x7f6efc0fe3a0 msgr2=0x7f6efc108af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:58:58.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 shutdown_connections 2024-04-26T01:58:58.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:58:58.580+0000 7f6f01000700 1 -- 172.21.15.37:0/970164073 wait complete. 2024-04-26T01:58:59.508 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '38b727e3', quiesce took 7.9 seconds. Will release after: 25.6 2024-04-26T01:59:00.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:58:59 smithi071 ceph-mon[32578]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:59:00.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:58:59 smithi037 ceph-mon[27906]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:59:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:58:59 smithi045 ceph-mon[32520]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:59:02.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:01 smithi071 ceph-mon[32578]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:59:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:01 smithi037 ceph-mon[27906]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:59:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:01 smithi045 ceph-mon[32520]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T01:59:04.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:03 smithi071 ceph-mon[32578]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T01:59:04.186 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:03 smithi037 ceph-mon[27906]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T01:59:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:03 smithi045 ceph-mon[32520]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T01:59:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.597+0000 7f8223200640 1 -- 172.21.15.37:0/3385951355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 msgr2=0x7f821c1065b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.597+0000 7f8223200640 1 --2- 172.21.15.37:0/3385951355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c1065b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8218009ca0 tx=0x7f821802a950 comp rx=0 tx=0).stop 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 -- 172.21.15.37:0/3385951355 shutdown_connections 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 --2- 172.21.15.37:0/3385951355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c1065b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 --2- 172.21.15.37:0/3385951355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c101640 0x7f821c105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 --2- 172.21.15.37:0/3385951355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f821c100d20 0x7f821c101100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 -- 172.21.15.37:0/3385951355 >> 172.21.15.37:0/3385951355 conn(0x7f821c0fc730 msgr2=0x7f821c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 -- 172.21.15.37:0/3385951355 shutdown_connections 2024-04-26T01:59:05.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.598+0000 7f8223200640 1 -- 172.21.15.37:0/3385951355 wait complete. 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.599+0000 7f8223200640 1 Processor -- start 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.599+0000 7f8223200640 1 -- start start 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8223200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 0x7f821c1a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8223200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f821c101640 0x7f821c1a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8223200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8223200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f821c118570 con 0x7f821c100d20 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8223200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1183f0 con 0x7f821c106120 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8223200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1186f0 con 0x7f821c101640 2024-04-26T01:59:05.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8221e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 0x7f821c1a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8221e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 0x7f821c1a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55128/0 (socket says 172.21.15.37:55128) 2024-04-26T01:59:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8221e00640 1 -- 172.21.15.37:0/472072295 learned_addr learned my addr 172.21.15.37:0/472072295 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8221400640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f821c101640 0x7f821c1a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.600+0000 7f8222800640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8222800640 1 -- 172.21.15.37:0/472072295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f821c101640 msgr2=0x7f821c1a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8222800640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f821c101640 0x7f821c1a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8222800640 1 -- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 msgr2=0x7f821c1a4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8222800640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 0x7f821c1a4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.602 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8222800640 1 -- 172.21.15.37:0/472072295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821c19f070 con 0x7f821c106120 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8221e00640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 0x7f821c1a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.601+0000 7f8222800640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c19e8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8218005cc0 tx=0x7f8218031f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.602+0000 7f8216a00640 1 -- 172.21.15.37:0/472072295 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218034040 con 0x7f821c106120 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.602+0000 7f8216a00640 1 -- 172.21.15.37:0/472072295 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821802fb80 con 0x7f821c106120 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.602+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f821c19f300 con 0x7f821c106120 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.602+0000 7f8216a00640 1 -- 172.21.15.37:0/472072295 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8218004490 con 0x7f821c106120 2024-04-26T01:59:05.603 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.603+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f821c1ab600 con 0x7f821c106120 2024-04-26T01:59:05.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.605+0000 7f8216a00640 1 -- 172.21.15.37:0/472072295 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f8218037020 con 0x7f821c106120 2024-04-26T01:59:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.607+0000 7f8216a00640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f82040a3e20 0x7f82040a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.607+0000 7f8227625740 1 Processor -- start 2024-04-26T01:59:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.607+0000 7f8216a00640 1 -- 172.21.15.37:0/472072295 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82180f1940 con 0x7f821c106120 2024-04-26T01:59:05.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.607+0000 7f8227625740 1 -- start start 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8221e00640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f82040a3e20 0x7f82040a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8227625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d60d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8227625740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d6b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8227625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d6df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8227625740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555a7cc6900 con 0x5555a7d612d0 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8227625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5555a7c6cef0 con 0x5555a7d6bb90 2024-04-26T01:59:05.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8227625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5555a7cc28c0 con 0x5555a7d609b0 2024-04-26T01:59:05.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8221400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d6b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8221400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d6b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55144/0 (socket says 172.21.15.37:55144) 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.608+0000 7f8221400640 1 -- 172.21.15.37:0/1364775963 learned_addr learned my addr 172.21.15.37:0/1364775963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8222800640 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d6df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8221e00640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f82040a3e20 0x7f82040a62e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f821c19fa30 tx=0x7f8210005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8221400640 1 -- 172.21.15.37:0/1364775963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 msgr2=0x5555a7d60d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8221400640 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d60d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8221400640 1 -- 172.21.15.37:0/1364775963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 msgr2=0x5555a7d6df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8221400640 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d6df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.609+0000 7f8221400640 1 -- 172.21.15.37:0/1364775963 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a79d9210 con 0x5555a7d612d0 2024-04-26T01:59:05.609 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.610+0000 7f8221400640 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d6b650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f820c009c10 tx=0x7f820c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aec1de3fb8c77d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.610+0000 7f820b000640 1 -- 172.21.15.37:0/1364775963 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c004050 con 0x5555a7d612d0 2024-04-26T01:59:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.610+0000 7f820b000640 1 -- 172.21.15.37:0/1364775963 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f820c0054d0 con 0x5555a7d612d0 2024-04-26T01:59:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.610+0000 7f820b000640 1 -- 172.21.15.37:0/1364775963 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820c004d00 con 0x5555a7d612d0 2024-04-26T01:59:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 -- 172.21.15.37:0/1364775963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 msgr2=0x5555a7d6b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.610 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d6b650 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f820c009c10 tx=0x7f820c011810 comp rx=0 tx=0).stop 2024-04-26T01:59:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 -- 172.21.15.37:0/1364775963 shutdown_connections 2024-04-26T01:59:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d6df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d6b650 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 --2- 172.21.15.37:0/1364775963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d60d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 -- 172.21.15.37:0/1364775963 >> 172.21.15.37:0/1364775963 conn(0x5555a7d5cec0 msgr2=0x5555a7d5d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:05.611 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 -- 172.21.15.37:0/1364775963 shutdown_connections 2024-04-26T01:59:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.611+0000 7f8227625740 1 -- 172.21.15.37:0/1364775963 wait complete. 2024-04-26T01:59:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.612+0000 7f8227625740 1 -- start start 2024-04-26T01:59:05.612 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.612+0000 7f8227625740 1 Processor -- start 2024-04-26T01:59:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d68b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d69070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5555a7d6e7c0 con 0x5555a7d612d0 2024-04-26T01:59:05.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5555a7c80310 con 0x5555a7d6bb90 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5555a7d6e940 con 0x5555a7d609b0 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8227625740 10 client.0 _ll_register_callbacks cb 0x5555a7c4ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8222800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8222800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:39204/0 (socket says 172.21.15.37:39204) 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8222800640 1 -- 172.21.15.37:0/3201997816 learned_addr learned my addr 172.21.15.37:0/3201997816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8221e00640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d68b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8221400640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d69070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.613+0000 7f8222800640 1 -- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 msgr2=0x5555a7d68b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.614+0000 7f8222800640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d68b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.614+0000 7f8222800640 1 -- 172.21.15.37:0/3201997816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 msgr2=0x5555a7d69070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.614+0000 7f8222800640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d69070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.614+0000 7f8222800640 1 -- 172.21.15.37:0/3201997816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a7aa5620 con 0x5555a7d6bb90 2024-04-26T01:59:05.614 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.614+0000 7f8221400640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d69070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f8222800640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d695b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82180bd700 tx=0x7f82180bdf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c186bc17e335bc4a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f81ff400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f81ff400640 1 -- 172.21.15.37:0/3201997816 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180c63e0 con 0x5555a7d6bb90 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f81ff400640 1 -- 172.21.15.37:0/3201997816 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82180b9de0 con 0x5555a7d6bb90 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f820c0054d0 con 0x5555a7d6bb90 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f81ff400640 1 -- 172.21.15.37:0/3201997816 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8218003b70 con 0x5555a7d6bb90 2024-04-26T01:59:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f8227625740 10 client.25190 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5555a7d6e7c0 con 0x5555a7d6bb90 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.616+0000 7f81ff400640 1 -- 172.21.15.37:0/3201997816 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f820c0054d0 con 0x5555a7d6bb90 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f81ff400640 1 -- 172.21.15.37:0/3201997816 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f8218030af0 con 0x5555a7d6bb90 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8227625740 10 client.25190 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8227625740 10 client.25190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8227625740 20 client.25190 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8227625740 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7d7be90 0x5555a7d7e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8227625740 4 client.25190 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555a7cda8b0 con 0x5555a7d7be90 2024-04-26T01:59:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.617+0000 7f8221e00640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7d7be90 0x5555a7d7e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.618+0000 7f8221e00640 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7d7be90 0x5555a7d7e2b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=152a199fa9485df7 server_cookie=de2768cbe2082355 in_seq=0 out_seq=0 2024-04-26T01:59:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.618+0000 7f81ff400640 10 client.25190 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.619+0000 7f81ff400640 1 -- 172.21.15.37:0/3201997816 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555a7cda8b0 con 0x5555a7d7be90 2024-04-26T01:59:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.619+0000 7f81ff400640 10 client.25190 handle_command_reply: tid=0 2024-04-26T01:59:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.619+0000 7f8227625740 1 client.25190 shutdown 2024-04-26T01:59:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 msgr2=0x5555a7d695b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d695b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f82180bd700 tx=0x7f82180bdf30 comp rx=0 tx=0).stop 2024-04-26T01:59:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 shutdown_connections 2024-04-26T01:59:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7d7be90 0x5555a7d7e2b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d6bb90 0x5555a7d695b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d612d0 0x5555a7d69070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 --2- 172.21.15.37:0/3201997816 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d609b0 0x5555a7d68b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 >> 172.21.15.37:0/3201997816 conn(0x5555a7d5cec0 msgr2=0x5555a7d62bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f81ff400640 0 client.25190 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 shutdown_connections 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 1 -- 172.21.15.37:0/3201997816 wait complete. 2024-04-26T01:59:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.621+0000 7f8227625740 20 client.25190 trim_cache size 0 max 16384 2024-04-26T01:59:05.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.637+0000 7f8227625740 1 Processor -- start 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.637+0000 7f8227625740 1 -- start start 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8227625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 0x5555a7d77680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8227625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d63370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8227625740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d5bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8227625740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b1ea0 con 0x5555a7d6bb90 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8227625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b4720 con 0x5555a7d609b0 2024-04-26T01:59:05.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8227625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b2510 con 0x5555a7d612d0 2024-04-26T01:59:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8222800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d5bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8221e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 0x5555a7d77680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8221e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 0x5555a7d77680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:39218/0 (socket says 172.21.15.37:39218) 2024-04-26T01:59:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8221e00640 1 -- 172.21.15.37:0/2490343924 learned_addr learned my addr 172.21.15.37:0/2490343924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:05.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.638+0000 7f8221400640 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d63370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221e00640 1 -- 172.21.15.37:0/2490343924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 msgr2=0x5555a7d63370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221e00640 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d63370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221e00640 1 -- 172.21.15.37:0/2490343924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 msgr2=0x5555a7d5bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221e00640 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d5bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221e00640 1 -- 172.21.15.37:0/2490343924 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555a7d61100 con 0x5555a7d609b0 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8222800640 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d5bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221400640 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d63370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.639+0000 7f8221e00640 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 0x5555a7d77680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5555a7d7eb30 tx=0x7f8210013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1189cc0e95413eb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f820a600640 1 -- 172.21.15.37:0/2490343924 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8210030070 con 0x5555a7d609b0 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f820a600640 1 -- 172.21.15.37:0/2490343924 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821002a4e0 con 0x5555a7d609b0 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f820a600640 1 -- 172.21.15.37:0/2490343924 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821000fa80 con 0x5555a7d609b0 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 -- 172.21.15.37:0/2490343924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 msgr2=0x5555a7d77680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 0x5555a7d77680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5555a7d7eb30 tx=0x7f8210013040 comp rx=0 tx=0).stop 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 -- 172.21.15.37:0/2490343924 shutdown_connections 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d5bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d63370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 --2- 172.21.15.37:0/2490343924 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d609b0 0x5555a7d77680 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x5555a7d7eb30 tx=0x7f8210013040 comp rx=0 tx=0).stop 2024-04-26T01:59:05.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.640+0000 7f8227625740 1 -- 172.21.15.37:0/2490343924 >> 172.21.15.37:0/2490343924 conn(0x5555a7d5cec0 msgr2=0x5555a7d5b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.641+0000 7f8227625740 1 -- 172.21.15.37:0/2490343924 shutdown_connections 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.641+0000 7f8227625740 1 -- 172.21.15.37:0/2490343924 wait complete. 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.641+0000 7f8227625740 1 -- start start 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.641+0000 7f8227625740 1 Processor -- start 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d715c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d71b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d72040 0x5555a7de1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b2be0 con 0x5555a7d6bb90 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b4050 con 0x5555a7d72040 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f821c1b32b0 con 0x5555a7d612d0 2024-04-26T01:59:05.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8227625740 10 client.0 _ll_register_callbacks cb 0x7f82180bd0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8221400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d71b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8221400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d71b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55166/0 (socket says 172.21.15.37:55166) 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8221400640 1 -- 172.21.15.37:0/3777004541 learned_addr learned my addr 172.21.15.37:0/3777004541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8222800640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d72040 0x5555a7de1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.642+0000 7f8221e00640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d715c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8222800640 1 -- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 msgr2=0x5555a7d715c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8222800640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d715c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8222800640 1 -- 172.21.15.37:0/3777004541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 msgr2=0x5555a7d71b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8222800640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d71b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8222800640 1 -- 172.21.15.37:0/3777004541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555a7de1980 con 0x5555a7d72040 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8221e00640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8221400640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d71b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8222800640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d72040 0x5555a7de1440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8218006010 tx=0x7f82180099f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88502a92ecf7dedc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f81ff400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:59:05.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f81ff400640 1 -- 172.21.15.37:0/3777004541 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82180c63e0 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f81ff400640 1 -- 172.21.15.37:0/3777004541 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f82180b95b0 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.643+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f821002a4e0 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f81ff400640 1 -- 172.21.15.37:0/3777004541 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8218003d10 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 10 client.25196 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5555a7de1980 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f81ff400640 1 -- 172.21.15.37:0/3777004541 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f821002a4e0 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f81ff400640 1 -- 172.21.15.37:0/3777004541 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f82180a3070 con 0x5555a7d72040 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 10 client.25196 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 10 client.25196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 20 client.25196 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7de2e70 0x5555a7de5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 4 client.25196 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.644+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555a7cd4320 con 0x5555a7de2e70 2024-04-26T01:59:05.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.645+0000 7f8221e00640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7de2e70 0x5555a7de5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.645+0000 7f8221e00640 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7de2e70 0x5555a7de5260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c351cfe7d30ce841 server_cookie=bf179c16d1b5b4da in_seq=0 out_seq=0 2024-04-26T01:59:05.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.645+0000 7f81ff400640 10 client.25196 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.655+0000 7f81ff400640 1 -- 172.21.15.37:0/3777004541 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555a7cd4320 con 0x5555a7de2e70 2024-04-26T01:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.655+0000 7f81ff400640 10 client.25196 handle_command_reply: tid=0 2024-04-26T01:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.655+0000 7f8227625740 1 client.25196 shutdown 2024-04-26T01:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.655+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d72040 msgr2=0x5555a7de1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.655+0000 7f8227625740 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d72040 0x5555a7de1440 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8218006010 tx=0x7f82180099f0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 shutdown_connections 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5555a7de2e70 0x5555a7de5260 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5555a7d72040 0x5555a7de1440 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5555a7d6bb90 0x5555a7d71b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 --2- 172.21.15.37:0/3777004541 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5555a7d612d0 0x5555a7d715c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f81ff400640 0 client.25196 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 >> 172.21.15.37:0/3777004541 conn(0x5555a7d5cec0 msgr2=0x5555a7d5b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 shutdown_connections 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 1 -- 172.21.15.37:0/3777004541 wait complete. 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.656+0000 7f8227625740 20 client.25196 trim_cache size 0 max 16384 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:59:05.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f82040a3e20 msgr2=0x7f82040a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f82040a3e20 0x7f82040a62e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f821c19fa30 tx=0x7f8210005f90 comp rx=0 tx=0).stop 2024-04-26T01:59:05.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 msgr2=0x7f821c19e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c19e8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8218005cc0 tx=0x7f8218031f40 comp rx=0 tx=0).stop 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 shutdown_connections 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f82040a3e20 0x7f82040a62e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f821c106120 0x7f821c19e8f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f821c101640 0x7f821c1a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 --2- 172.21.15.37:0/472072295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f821c100d20 0x7f821c1a4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 >> 172.21.15.37:0/472072295 conn(0x7f821c0fc730 msgr2=0x7f821c0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 shutdown_connections 2024-04-26T01:59:05.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:05.657+0000 7f8223200640 1 -- 172.21.15.37:0/472072295 wait complete. 2024-04-26T01:59:06.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:05 smithi071 ceph-mon[32578]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-26T01:59:06.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:05 smithi037 ceph-mon[27906]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-26T01:59:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:05 smithi045 ceph-mon[32520]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-26T01:59:07.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:06 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:07.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:06 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:07.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:06 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T01:59:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:07 smithi071 ceph-mon[32578]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-26T01:59:08.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:07 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:59:08.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:07 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:59:08.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:07 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:59:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:07 smithi037 ceph-mon[27906]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-26T01:59:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:07 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:59:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:07 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:59:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:07 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:59:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:07 smithi045 ceph-mon[32520]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 16 op/s 2024-04-26T01:59:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:07 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T01:59:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:07 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T01:59:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:07 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T01:59:10.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:09 smithi071 ceph-mon[32578]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T01:59:10.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:09 smithi037 ceph-mon[27906]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T01:59:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:09 smithi045 ceph-mon[32520]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T01:59:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:11 smithi071 ceph-mon[32578]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T01:59:12.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:11 smithi037 ceph-mon[27906]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T01:59:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:11 smithi045 ceph-mon[32520]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2024-04-26T01:59:14.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:13 smithi071 ceph-mon[32578]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T01:59:14.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:13 smithi037 ceph-mon[27906]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T01:59:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:13 smithi045 ceph-mon[32520]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2024-04-26T01:59:16.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:15 smithi071 ceph-mon[32578]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:15 smithi037 ceph-mon[27906]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:15 smithi045 ceph-mon[32520]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:17.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:17 smithi037 ceph-mon[27906]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:17 smithi071 ceph-mon[32578]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:17 smithi045 ceph-mon[32520]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:18.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.799+0000 7fdd51200640 1 -- 172.21.15.37:0/3635196397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 msgr2=0x7fdd4c103310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.799+0000 7fdd51200640 1 --2- 172.21.15.37:0/3635196397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 0x7fdd4c103310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009be0 tx=0x7fdd3c02a750 comp rx=0 tx=0).stop 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 -- 172.21.15.37:0/3635196397 shutdown_connections 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 --2- 172.21.15.37:0/3635196397 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c10c350 0x7fdd4c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 --2- 172.21.15.37:0/3635196397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c103850 0x7fdd4c10bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 --2- 172.21.15.37:0/3635196397 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 0x7fdd4c103310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 -- 172.21.15.37:0/3635196397 >> 172.21.15.37:0/3635196397 conn(0x7fdd4c0fc730 msgr2=0x7fdd4c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 -- 172.21.15.37:0/3635196397 shutdown_connections 2024-04-26T01:59:18.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.800+0000 7fdd51200640 1 -- 172.21.15.37:0/3635196397 wait complete. 2024-04-26T01:59:18.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 Processor -- start 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 -- start start 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 0x7fdd4c198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c103850 0x7fdd4c198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 0x7fdd4c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c111fe0 con 0x7fdd4c10c350 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c111e60 con 0x7fdd4c102f30 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.801+0000 7fdd51200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c112160 con 0x7fdd4c103850 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 0x7fdd4c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c103850 0x7fdd4c198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c103850 0x7fdd4c198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:48652/0 (socket says 172.21.15.37:48652) 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 0x7fdd4c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45472/0 (socket says 172.21.15.37:45472) 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd42a00640 1 -- 172.21.15.37:0/1616123914 learned_addr learned my addr 172.21.15.37:0/1616123914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:18.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4b400640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 0x7fdd4c198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 -- 172.21.15.37:0/1616123914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c103850 msgr2=0x7fdd4c198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c103850 0x7fdd4c198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 -- 172.21.15.37:0/1616123914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 msgr2=0x7fdd4c198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 0x7fdd4c198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.802+0000 7fdd4be00640 1 -- 172.21.15.37:0/1616123914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c19d530 con 0x7fdd4c10c350 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.803+0000 7fdd4be00640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 0x7fdd4c19ce00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e970 tx=0x7fdd4400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.803+0000 7fdd48c00640 1 -- 172.21.15.37:0/1616123914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400ccc0 con 0x7fdd4c10c350 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.803+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c19d820 con 0x7fdd4c10c350 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.803+0000 7fdd48c00640 1 -- 172.21.15.37:0/1616123914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd44005020 con 0x7fdd4c10c350 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.803+0000 7fdd48c00640 1 -- 172.21.15.37:0/1616123914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44004a00 con 0x7fdd4c10c350 2024-04-26T01:59:18.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.803+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c1a50b0 con 0x7fdd4c10c350 2024-04-26T01:59:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.806+0000 7fdd48c00640 1 -- 172.21.15.37:0/1616123914 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fdd44004ba0 con 0x7fdd4c10c350 2024-04-26T01:59:18.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.806+0000 7fdd55322740 1 Processor -- start 2024-04-26T01:59:18.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 -- start start 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 0x55ad3ff52c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff61a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff64390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ad3feb8900 con 0x55ad3ff528b0 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad3fe5eef0 con 0x55ad3ff531d0 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd55322740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ad3feb48c0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.807+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 0x55ad3ff52c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 0x55ad3ff52c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45488/0 (socket says 172.21.15.37:45488) 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 -- 172.21.15.37:0/1669863856 learned_addr learned my addr 172.21.15.37:0/1669863856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd4b400640 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff61a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff64390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 -- 172.21.15.37:0/1669863856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 msgr2=0x55ad3ff64390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff64390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 -- 172.21.15.37:0/1669863856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 msgr2=0x55ad3ff61a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff61a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 -- 172.21.15.37:0/1669863856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3fbcb210 con 0x55ad3ff528b0 2024-04-26T01:59:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.808+0000 7fdd42a00640 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 0x55ad3ff52c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009fb0 tx=0x7fdd34009910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7319469337c622 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd40c00640 1 -- 172.21.15.37:0/1669863856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd34004030 con 0x55ad3ff528b0 2024-04-26T01:59:18.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd40c00640 1 -- 172.21.15.37:0/1669863856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd340035f0 con 0x55ad3ff528b0 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd48c00640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fdd300a3e20 0x7fdd300a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd40c00640 1 -- 172.21.15.37:0/1669863856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd3400f890 con 0x55ad3ff528b0 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd48c00640 1 -- 172.21.15.37:0/1616123914 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fdd440d0960 con 0x7fdd4c10c350 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd4b400640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fdd300a3e20 0x7fdd300a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd55322740 1 -- 172.21.15.37:0/1669863856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 msgr2=0x55ad3ff52c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.809+0000 7fdd55322740 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 0x55ad3ff52c90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdd34009fb0 tx=0x7fdd34009910 comp rx=0 tx=0).stop 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 -- 172.21.15.37:0/1669863856 shutdown_connections 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff64390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff61a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 --2- 172.21.15.37:0/1669863856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff528b0 0x55ad3ff52c90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 -- 172.21.15.37:0/1669863856 >> 172.21.15.37:0/1669863856 conn(0x55ad3ff4eec0 msgr2=0x55ad3ff4f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 -- 172.21.15.37:0/1669863856 shutdown_connections 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 -- 172.21.15.37:0/1669863856 wait complete. 2024-04-26T01:59:18.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd4b400640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fdd300a3e20 0x7fdd300a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000f80 tx=0x7fdd3c005fd0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 -- start start 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.810+0000 7fdd55322740 1 Processor -- start 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 0x55ad3ff5ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff5f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ad3ff64bd0 con 0x55ad3ff531d0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55ad3fe72310 con 0x55ad3ff528b0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55ad3ff64d50 con 0x55ad3ff61fa0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd55322740 10 client.0 _ll_register_callbacks cb 0x55ad3fe40cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd4b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 0x55ad3ff5ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd4b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 0x55ad3ff5ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:57160/0 (socket says 172.21.15.37:57160) 2024-04-26T01:59:18.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd4b400640 1 -- 172.21.15.37:0/2912516246 learned_addr learned my addr 172.21.15.37:0/2912516246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.811+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff5f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4be00640 1 -- 172.21.15.37:0/2912516246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 msgr2=0x55ad3ff5ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 0x55ad3ff5ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4be00640 1 -- 172.21.15.37:0/2912516246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff531d0 msgr2=0x55ad3ff5f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4be00640 1 -- 172.21.15.37:0/2912516246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3fc97620 con 0x55ad3ff61fa0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 0x55ad3ff5ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff5f8b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd440cdf00 tx=0x7fdd4409c0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=821b7183a7076b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd39600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd39600640 1 -- 172.21.15.37:0/2912516246 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd4401c040 con 0x55ad3ff61fa0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd39600640 1 -- 172.21.15.37:0/2912516246 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fdd440a4e60 con 0x55ad3ff61fa0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.812+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd340035f0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd39600640 1 -- 172.21.15.37:0/2912516246 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdd44087070 con 0x55ad3ff61fa0 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 10 client.25252 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ad3ff70e10 con 0x55ad3ff61fa0 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd39600640 1 -- 172.21.15.37:0/2912516246 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd340035f0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd39600640 1 -- 172.21.15.37:0/2912516246 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fdd44085ce0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 10 client.25252 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 10 client.25252 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 20 client.25252 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ff72280 0x55ad3ff74670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 4 client.25252 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:18.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.813+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad3fecc8b0 con 0x55ad3ff72280 2024-04-26T01:59:18.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.814+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ff72280 0x55ad3ff74670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.814+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ff72280 0x55ad3ff74670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c245e8f1eebafd server_cookie=1f9661c41580325b in_seq=0 out_seq=0 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.814+0000 7fdd39600640 10 client.25252 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd39600640 1 -- 172.21.15.37:0/2912516246 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad3fecc8b0 con 0x55ad3ff72280 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd39600640 10 client.25252 handle_command_reply: tid=0 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 client.25252 shutdown 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 msgr2=0x55ad3ff5f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff5f8b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdd440cdf00 tx=0x7fdd4409c0a0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 shutdown_connections 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ff72280 0x55ad3ff74670 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff5f8b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 --2- 172.21.15.37:0/2912516246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff528b0 0x55ad3ff5ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 >> 172.21.15.37:0/2912516246 conn(0x55ad3ff4eec0 msgr2=0x55ad3ff54d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:18.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.815+0000 7fdd39600640 0 client.25252 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.816+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 shutdown_connections 2024-04-26T01:59:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.816+0000 7fdd55322740 1 -- 172.21.15.37:0/2912516246 wait complete. 2024-04-26T01:59:18.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.816+0000 7fdd55322740 20 client.25252 trim_cache size 0 max 16384 2024-04-26T01:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 Processor -- start 2024-04-26T01:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 -- start start 2024-04-26T01:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff6da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff79fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1a9760 con 0x55ad3ff61fa0 2024-04-26T01:59:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1abfe0 con 0x55ad3ff531d0 2024-04-26T01:59:18.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd55322740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1a9dd0 con 0x55ad3ff528b0 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff79fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff79fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45510/0 (socket says 172.21.15.37:45510) 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 -- 172.21.15.37:0/264180914 learned_addr learned my addr 172.21.15.37:0/264180914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.832+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4b400640 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff6da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 -- 172.21.15.37:0/264180914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 msgr2=0x55ad3ff6da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff6da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 -- 172.21.15.37:0/264180914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 msgr2=0x55ad3ff5de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 -- 172.21.15.37:0/264180914 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad3ff74cf0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd42a00640 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd4be00640 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff79fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b9040 tx=0x7fdd4409c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc49ff898b2b71b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.833+0000 7fdd3be00640 1 -- 172.21.15.37:0/264180914 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440a0a30 con 0x55ad3ff61fa0 2024-04-26T01:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.834+0000 7fdd3be00640 1 -- 172.21.15.37:0/264180914 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd44087030 con 0x55ad3ff61fa0 2024-04-26T01:59:18.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.834+0000 7fdd3be00640 1 -- 172.21.15.37:0/264180914 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440984f0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.834+0000 7fdd55322740 1 -- 172.21.15.37:0/264180914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 msgr2=0x55ad3ff79fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.834+0000 7fdd55322740 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff79fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdd440b9040 tx=0x7fdd4409c800 comp rx=0 tx=0).stop 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.834+0000 7fdd55322740 1 -- 172.21.15.37:0/264180914 shutdown_connections 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ff79fd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff5de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 --2- 172.21.15.37:0/264180914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff6da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 -- 172.21.15.37:0/264180914 >> 172.21.15.37:0/264180914 conn(0x55ad3ff4eec0 msgr2=0x55ad3ff5d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 -- 172.21.15.37:0/264180914 shutdown_connections 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 -- 172.21.15.37:0/264180914 wait complete. 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 -- start start 2024-04-26T01:59:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.835+0000 7fdd55322740 1 Processor -- start 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff67570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff67ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ffcf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1aa4a0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1ab910 con 0x55ad3ff531d0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c1aab70 con 0x55ad3ff528b0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd55322740 10 client.0 _ll_register_callbacks cb 0x7fdd4409c4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ffcf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd42a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff67ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ffcf800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45518/0 (socket says 172.21.15.37:45518) 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 -- 172.21.15.37:0/2377514433 learned_addr learned my addr 172.21.15.37:0/2377514433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff67570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 -- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 msgr2=0x55ad3ff67570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff67570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 -- 172.21.15.37:0/2377514433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 msgr2=0x55ad3ff67ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff67ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4be00640 1 -- 172.21.15.37:0/2377514433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad3ff59c30 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.836+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff67570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd4be00640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ffcf800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd440a4dc0 tx=0x7fdd44009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b0c4430788c034 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd39600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd39600640 1 -- 172.21.15.37:0/2377514433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd440ce450 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd39600640 1 -- 172.21.15.37:0/2377514433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fdd440b5e00 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd44087030 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd39600640 1 -- 172.21.15.37:0/2377514433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fdd440a52f0 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd55322740 10 client.15120 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55ad3ffd8220 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.837+0000 7fdd39600640 1 -- 172.21.15.37:0/2377514433 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd44087030 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd39600640 1 -- 172.21.15.37:0/2377514433 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fdd4409b540 con 0x55ad3ff61fa0 2024-04-26T01:59:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd55322740 10 client.15120 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd55322740 10 client.15120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd55322740 20 client.15120 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd55322740 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ffd86a0 0x55ad3ffdaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd55322740 4 client.15120 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad3fec6320 con 0x55ad3ffd86a0 2024-04-26T01:59:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.838+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ffd86a0 0x55ad3ffdaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.839+0000 7fdd4b400640 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ffd86a0 0x55ad3ffdaa90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cc836ee86014bd server_cookie=abda58f2e8a4ccb6 in_seq=0 out_seq=0 2024-04-26T01:59:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.839+0000 7fdd39600640 10 client.15120 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd39600640 1 -- 172.21.15.37:0/2377514433 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad3fec6320 con 0x55ad3ffd86a0 2024-04-26T01:59:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd39600640 10 client.15120 handle_command_reply: tid=0 2024-04-26T01:59:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 client.15120 shutdown 2024-04-26T01:59:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 msgr2=0x55ad3ffcf800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ffcf800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdd440a4dc0 tx=0x7fdd44009620 comp rx=0 tx=0).stop 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 shutdown_connections 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55ad3ffd86a0 0x55ad3ffdaa90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ad3ff61fa0 0x55ad3ffcf800 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55ad3ff531d0 0x55ad3ff67ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 --2- 172.21.15.37:0/2377514433 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55ad3ff528b0 0x55ad3ff67570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd39600640 0 client.15120 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 >> 172.21.15.37:0/2377514433 conn(0x55ad3ff4eec0 msgr2=0x55ad3ff4d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 shutdown_connections 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 1 -- 172.21.15.37:0/2377514433 wait complete. 2024-04-26T01:59:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.845+0000 7fdd55322740 20 client.15120 trim_cache size 0 max 16384 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fdd300a3e20 msgr2=0x7fdd300a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fdd300a3e20 0x7fdd300a62e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c000f80 tx=0x7fdd3c005fd0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 msgr2=0x7fdd4c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:18.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 0x7fdd4c19ce00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e970 tx=0x7fdd4400ee30 comp rx=0 tx=0).stop 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 shutdown_connections 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fdd300a3e20 0x7fdd300a62e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd4c10c350 0x7fdd4c19ce00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fdd4c103850 0x7fdd4c198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 --2- 172.21.15.37:0/1616123914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fdd4c102f30 0x7fdd4c198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 >> 172.21.15.37:0/1616123914 conn(0x7fdd4c0fc730 msgr2=0x7fdd4c0fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 shutdown_connections 2024-04-26T01:59:18.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:18.846+0000 7fdd51200640 1 -- 172.21.15.37:0/1616123914 wait complete. 2024-04-26T01:59:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:19 smithi071 ceph-mon[32578]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:59:20.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:19 smithi037 ceph-mon[27906]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:59:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:19 smithi045 ceph-mon[32520]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-04-26T01:59:20.350 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 62% (7810/12492) Updating files: 63% (7870/12492) Updating files: 64% (7995/12492) Updating files: 65% (8120/12492) Updating files: 66% (8245/12492) Updating files: 67% (8370/12492) Updating files: 68% (8495/12492) Updating files: 69% (8620/12492) Updating files: 70% (8745/12492) Updating files: 71% (8870/12492) Updating files: 72% (8995/12492) Updating files: 73% (9120/12492) Updating files: 74% (9245/12492) Updating files: 75% (9369/12492) Updating files: 76% (9494/12492) Updating files: 77% (9619/12492) Updating files: 78% (9744/12492) Updating files: 79% (9869/12492) Updating files: 80% (9994/12492) Updating files: 81% (10119/12492) Updating files: 82% (10244/12492) Updating files: 83% (10369/12492) Updating files: 84% (10494/12492) Updating files: 85% (10619/12492) Updating files: 86% (10744/12492) Updating files: 87% (10869/12492) Updating files: 88% (10993/12492) Updating files: 89% (11118/12492) Updating files: 90% (11243/12492) Updating files: 91% (11368/12492) Updating files: 92% (11493/12492) Updating files: 93% (11618/12492) Updating files: 94% (11743/12492) Updating files: 95% (11868/12492) Updating files: 96% (11993/12492) Updating files: 97% (12118/12492) Updating files: 98% (12243/12492) Updating files: 99% (12368/12492) Updating files: 100% (12492/12492) Updating files: 100% (12492/12492), done. 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to 'afa1933b0fdbb6c99c947d1eda34d661d23cd327'. 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-04-26T01:59:20.628 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-04-26T01:59:20.629 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at afa1933b0fd qa/suites/fs: add quiescer to the fs suite 2024-04-26T01:59:20.632 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T01:59:20.749 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T01:59:20.758 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T01:59:20.758 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T01:59:21.583 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T01:59:21.707 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T01:59:21.800 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T01:59:21.805 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T01:59:21.805 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T01:59:21.895 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T01:59:21.905 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T01:59:21.905 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T01:59:21.958 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-04-26T01:59:21.959 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-04-26T01:59:21.960 DEBUG:teuthology.orchestra.run.smithi037:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=afa1933b0fdbb6c99c947d1eda34d661d23cd327 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-04-26T01:59:22.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:21 smithi071 ceph-mon[32578]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T01:59:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:21 smithi037 ceph-mon[27906]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T01:59:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:21 smithi045 ceph-mon[32520]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T01:59:24.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:23 smithi071 ceph-mon[32578]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T01:59:24.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:23 smithi037 ceph-mon[27906]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T01:59:24.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:23 smithi045 ceph-mon[32520]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 511 B/s wr, 3 op/s 2024-04-26T01:59:25.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T01:59:25.583 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:25.985 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:25 smithi037 ceph-mon[27906]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:59:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:25 smithi071 ceph-mon[32578]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:59:26.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:25 smithi045 ceph-mon[32520]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:59:27.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:27 smithi037 ceph-mon[27906]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 -- 172.21.15.37:0/368840054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c103450 msgr2=0x7f854c10f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 --2- 172.21.15.37:0/368840054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c103450 0x7f854c10f8b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f854400b0a0 tx=0x7f854400b3b0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 -- 172.21.15.37:0/368840054 shutdown_connections 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 --2- 172.21.15.37:0/368840054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c103450 0x7f854c10f8b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 --2- 172.21.15.37:0/368840054 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f854c108a50 0x7f854c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 --2- 172.21.15.37:0/368840054 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 -- 172.21.15.37:0/368840054 >> 172.21.15.37:0/368840054 conn(0x7f854c079ba0 msgr2=0x7f854c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 -- 172.21.15.37:0/368840054 shutdown_connections 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.018+0000 7f8553e00700 1 -- 172.21.15.37:0/368840054 wait complete. 2024-04-26T01:59:28.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.019+0000 7f8553e00700 1 Processor -- start 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.019+0000 7f8553e00700 1 -- start start 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8553e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8553e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f854c103450 0x7f854c199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8553e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c108a50 0x7f854c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8553e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112c50 con 0x7f854c108a50 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8553e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112ad0 con 0x7f854c102a90 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8553e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f854c112dd0 con 0x7f854c103450 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54684/0 (socket says 172.21.15.37:54684) 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c108a50 0x7f854c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 -- 172.21.15.37:0/3380756651 learned_addr learned my addr 172.21.15.37:0/3380756651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8550a00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f854c103450 0x7f854c199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:28.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 -- 172.21.15.37:0/3380756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f854c103450 msgr2=0x7f854c199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f854c103450 0x7f854c199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 -- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c108a50 msgr2=0x7f854c19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c108a50 0x7f854c19d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.020+0000 7f8551400700 1 -- 172.21.15.37:0/3380756651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c19e110 con 0x7f854c102a90 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.021+0000 7f8551400700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c199100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f854000e9a0 tx=0x7f854000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.021+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f854000ca10 con 0x7f854c102a90 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.021+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c19e400 con 0x7f854c102a90 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.021+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f854000cbb0 con 0x7f854c102a90 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.021+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8540018680 con 0x7f854c102a90 2024-04-26T01:59:28.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.021+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c074c70 con 0x7f854c102a90 2024-04-26T01:59:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.022+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c069730 con 0x7f854c102a90 2024-04-26T01:59:28.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.024+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f8540010430 con 0x7f854c102a90 2024-04-26T01:59:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.026+0000 7f854a000700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f85340a3f50 0x7f85340a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.026+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f85400d0b80 con 0x7f854c102a90 2024-04-26T01:59:28.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.026+0000 7f8550a00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f85340a3f50 0x7f85340a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:28.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.027+0000 7f8550a00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f85340a3f50 0x7f85340a6410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8538009680 tx=0x7f8538012040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:28.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.030+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f854009cfa0 con 0x7f854c102a90 2024-04-26T01:59:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:27 smithi071 ceph-mon[32578]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:59:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:27 smithi045 ceph-mon[32520]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 426 B/s wr, 2 op/s 2024-04-26T01:59:28.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.313+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f854c19a490 con 0x7f854c102a90 2024-04-26T01:59:28.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.314+0000 7f854a000700 1 -- 172.21.15.37:0/3380756651 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f854c19a490 con 0x7f854c102a90 2024-04-26T01:59:28.313 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T01:59:28.313 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T01:59:28.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f85340a3f50 msgr2=0x7f85340a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:28.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f85340a3f50 0x7f85340a6410 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8538009680 tx=0x7f8538012040 comp rx=0 tx=0).stop 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 msgr2=0x7f854c199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c199100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f854000e9a0 tx=0x7f854000ecb0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 shutdown_connections 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f854c108a50 0x7f854c19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f85340a3f50 0x7f85340a6410 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f854c102a90 0x7f854c199100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 --2- 172.21.15.37:0/3380756651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f854c103450 0x7f854c199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.317+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 >> 172.21.15.37:0/3380756651 conn(0x7f854c079ba0 msgr2=0x7f854c101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:28.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.318+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 shutdown_connections 2024-04-26T01:59:28.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:28.318+0000 7f8553e00700 1 -- 172.21.15.37:0/3380756651 wait complete. 2024-04-26T01:59:28.318 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T01:59:29.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:28 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3380756651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:29.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:28 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3380756651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:29.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:28 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3380756651' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T01:59:29.257 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db --set-id 38b727e3 --release --await' 2024-04-26T01:59:29.258 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db --set-id 38b727e3 --release --await 2024-04-26T01:59:29.733 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T01:59:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:29 smithi071 ceph-mon[32578]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:29 smithi071 ceph-mon[32578]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T01:59:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:29 smithi037 ceph-mon[27906]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:29 smithi037 ceph-mon[27906]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T01:59:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:29 smithi045 ceph-mon[32520]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T01:59:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:29 smithi045 ceph-mon[32520]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-26T01:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 -- 172.21.15.37:0/1178672295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 msgr2=0x7faf141047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 --2- 172.21.15.37:0/1178672295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141047a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf04009900 tx=0x7faf0402a410 comp rx=0 tx=0).stop 2024-04-26T01:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 -- 172.21.15.37:0/1178672295 shutdown_connections 2024-04-26T01:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 --2- 172.21.15.37:0/1178672295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf14109490 0x7faf14111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 --2- 172.21.15.37:0/1178672295 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf14104d70 0x7faf14108db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 --2- 172.21.15.37:0/1178672295 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141047a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.998+0000 7faf1b000640 1 -- 172.21.15.37:0/1178672295 >> 172.21.15.37:0/1178672295 conn(0x7faf140ffc40 msgr2=0x7faf14102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.999+0000 7faf1b000640 1 -- 172.21.15.37:0/1178672295 shutdown_connections 2024-04-26T01:59:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:31.999+0000 7faf1b000640 1 -- 172.21.15.37:0/1178672295 wait complete. 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 Processor -- start 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 -- start start 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf14109490 0x7faf141a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1419a7e0 0x7faf1419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf14114420 con 0x7faf14109490 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf141142a0 con 0x7faf1419a7e0 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.000+0000 7faf1b000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf141145a0 con 0x7faf141043c0 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:32814/0 (socket says 172.21.15.37:32814) 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf13e00640 1 -- 172.21.15.37:0/4158810395 learned_addr learned my addr 172.21.15.37:0/4158810395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf19000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1419a7e0 0x7faf1419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf13400640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf14109490 0x7faf141a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf19000640 1 -- 172.21.15.37:0/4158810395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 msgr2=0x7faf141a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf19000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf19000640 1 -- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf14109490 msgr2=0x7faf141a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf19000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf14109490 0x7faf141a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf19000640 1 -- 172.21.15.37:0/4158810395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1419b370 con 0x7faf1419a7e0 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.001+0000 7faf13e00640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:32.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.002+0000 7faf19000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1419a7e0 0x7faf1419ac90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009fd0 tx=0x7faf0c00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.002+0000 7faf10c00640 1 -- 172.21.15.37:0/4158810395 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c00ce20 con 0x7faf1419a7e0 2024-04-26T01:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.002+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1419b660 con 0x7faf1419a7e0 2024-04-26T01:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.002+0000 7faf10c00640 1 -- 172.21.15.37:0/4158810395 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c005020 con 0x7faf1419a7e0 2024-04-26T01:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.002+0000 7faf10c00640 1 -- 172.21.15.37:0/4158810395 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c010690 con 0x7faf1419a7e0 2024-04-26T01:59:32.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.002+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a77f0 con 0x7faf1419a7e0 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf10c00640 1 -- 172.21.15.37:0/4158810395 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf0c0054d0 con 0x7faf1419a7e0 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 Processor -- start 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 -- start start 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98ba8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba9140 0x559f98bb34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f98b0e900 con 0x559f98bb3a00 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559f98ab4ef0 con 0x559f98ba8820 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf1de2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559f98b0a8c0 con 0x559f98ba9140 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.006+0000 7faf13400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98ba8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98ba8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:45012/0 (socket says 172.21.15.37:45012) 2024-04-26T01:59:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 -- 172.21.15.37:0/2541685007 learned_addr learned my addr 172.21.15.37:0/2541685007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13e00640 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba9140 0x559f98bb34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 -- 172.21.15.37:0/2541685007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba9140 msgr2=0x559f98bb34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba9140 0x559f98bb34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 -- 172.21.15.37:0/2541685007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 msgr2=0x559f98bb5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 -- 172.21.15.37:0/2541685007 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f98821210 con 0x559f98ba8820 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf10c00640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7faef80a3d10 0x7faef80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.007+0000 7faf13400640 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98ba8c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf000058c0 tx=0x7faf00005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dd1fd6d1364b795 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.008+0000 7faf10c00640 1 -- 172.21.15.37:0/4158810395 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7faf0c0d0eb0 con 0x7faf1419a7e0 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.008+0000 7faf13e00640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7faef80a3d10 0x7faef80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.008+0000 7faf09600640 1 -- 172.21.15.37:0/2541685007 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf00005020 con 0x559f98ba8820 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.008+0000 7faf09600640 1 -- 172.21.15.37:0/2541685007 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf000047a0 con 0x559f98ba8820 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.008+0000 7faf09600640 1 -- 172.21.15.37:0/2541685007 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0000fa00 con 0x559f98ba8820 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 -- 172.21.15.37:0/2541685007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 msgr2=0x559f98ba8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98ba8c00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf000058c0 tx=0x7faf00005850 comp rx=0 tx=0).stop 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf13e00640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7faef80a3d10 0x7faef80a61d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x559f98ba9dc0 tx=0x7faf04005a20 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 -- 172.21.15.37:0/2541685007 shutdown_connections 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba9140 0x559f98bb34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2541685007 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98ba8c00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 -- 172.21.15.37:0/2541685007 >> 172.21.15.37:0/2541685007 conn(0x559f98ba4d30 msgr2=0x559f98ba5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 -- 172.21.15.37:0/2541685007 shutdown_connections 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 -- 172.21.15.37:0/2541685007 wait complete. 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.009+0000 7faf1de2f740 1 -- start start 2024-04-26T01:59:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 Processor -- start 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba8820 0x559f98bb09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb0f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98bb1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559f98bb6630 con 0x559f98ba8820 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x559f98ac8310 con 0x559f98ba9140 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x559f98bb67b0 con 0x559f98bb3a00 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf1de2f740 10 client.0 _ll_register_callbacks cb 0x559f98a96cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.010+0000 7faf13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba8820 0x559f98bb09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf13e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba8820 0x559f98bb09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60456/0 (socket says 172.21.15.37:60456) 2024-04-26T01:59:32.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf13e00640 1 -- 172.21.15.37:0/785344053 learned_addr learned my addr 172.21.15.37:0/785344053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98bb1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf13400640 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 -- 172.21.15.37:0/785344053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 msgr2=0x559f98bb0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb0f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 -- 172.21.15.37:0/785344053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba8820 msgr2=0x559f98bb09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba8820 0x559f98bb09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 -- 172.21.15.37:0/785344053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f988ed620 con 0x559f98bb3a00 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faf19000640 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98bb1460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ce5a0 tx=0x7faf0c09ce20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=afc94da7d2f02c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faefe000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faefe000640 1 -- 172.21.15.37:0/785344053 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b6220 con 0x559f98bb3a00 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.011+0000 7faefe000640 1 -- 172.21.15.37:0/785344053 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7faf0c099490 con 0x559f98bb3a00 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf000047a0 con 0x559f98bb3a00 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faefe000640 1 -- 172.21.15.37:0/785344053 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7faf0c0a5600 con 0x559f98bb3a00 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 10 client.25276 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:32.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559f98bc2940 con 0x559f98bb3a00 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faefe000640 1 -- 172.21.15.37:0/785344053 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf000047a0 con 0x559f98bb3a00 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faefe000640 1 -- 172.21.15.37:0/785344053 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7faf0c098ce0 con 0x559f98bb3a00 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 10 client.25276 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 10 client.25276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 20 client.25276 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.012+0000 7faf1de2f740 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98bc3e00 0x559f98bc61f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.013+0000 7faf1de2f740 4 client.25276 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.013+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f98b228b0 con 0x559f98bc3e00 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.013+0000 7faf13e00640 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98bc3e00 0x559f98bc61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.013+0000 7faf13e00640 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98bc3e00 0x559f98bc61f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f514df05862c159e server_cookie=9a46ba5049cb7690 in_seq=0 out_seq=0 2024-04-26T01:59:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.013+0000 7faefe000640 10 client.25276 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.014+0000 7faefe000640 1 -- 172.21.15.37:0/785344053 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f98b228b0 con 0x559f98bc3e00 2024-04-26T01:59:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.014+0000 7faefe000640 10 client.25276 handle_command_reply: tid=0 2024-04-26T01:59:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.014+0000 7faf1de2f740 1 client.25276 shutdown 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 msgr2=0x559f98bb1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98bb1460 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf0c0ce5a0 tx=0x7faf0c09ce20 comp rx=0 tx=0).stop 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 shutdown_connections 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98bc3e00 0x559f98bc61f0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98bb1460 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faefe000640 0 client.25276 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 --2- 172.21.15.37:0/785344053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba8820 0x559f98bb09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 >> 172.21.15.37:0/785344053 conn(0x559f98ba4d30 msgr2=0x559f98baaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 shutdown_connections 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 1 -- 172.21.15.37:0/785344053 wait complete. 2024-04-26T01:59:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.015+0000 7faf1de2f740 20 client.25276 trim_cache size 0 max 16384 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.031+0000 7faf1de2f740 1 Processor -- start 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 -- start start 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98bbf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 0x559f98bab1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98ba3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf141add50 con 0x559f98ba9140 2024-04-26T01:59:32.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf141b05d0 con 0x559f98ba8820 2024-04-26T01:59:32.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf1de2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf141ae3c0 con 0x559f98bb3a00 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf13400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 0x559f98bab1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.032+0000 7faf19000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98ba3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf13e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98bbf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf13400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 0x559f98bab1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60458/0 (socket says 172.21.15.37:60458) 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf13400640 1 -- 172.21.15.37:0/812068941 learned_addr learned my addr 172.21.15.37:0/812068941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf19000640 1 -- 172.21.15.37:0/812068941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 msgr2=0x559f98bbf4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf19000640 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98bbf4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf19000640 1 -- 172.21.15.37:0/812068941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 msgr2=0x559f98bab1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf19000640 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 0x559f98bab1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf19000640 1 -- 172.21.15.37:0/812068941 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f98ba8f70 con 0x559f98bb3a00 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf13400640 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 0x559f98bab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf13e00640 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98bbf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.033+0000 7faf19000640 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98ba3d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c760 tx=0x7faf0c0990c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49ee99407765a094 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.034+0000 7faf08c00640 1 -- 172.21.15.37:0/812068941 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0b6240 con 0x559f98bb3a00 2024-04-26T01:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.034+0000 7faf08c00640 1 -- 172.21.15.37:0/812068941 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c099490 con 0x559f98bb3a00 2024-04-26T01:59:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.034+0000 7faf08c00640 1 -- 172.21.15.37:0/812068941 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0c098b70 con 0x559f98bb3a00 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.034+0000 7faf1de2f740 1 -- 172.21.15.37:0/812068941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 msgr2=0x559f98ba3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.034+0000 7faf1de2f740 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98ba3d10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c760 tx=0x7faf0c0990c0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 -- 172.21.15.37:0/812068941 shutdown_connections 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98bb3a00 0x559f98ba3d10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98ba9140 0x559f98bab1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 --2- 172.21.15.37:0/812068941 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba8820 0x559f98bbf4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 -- 172.21.15.37:0/812068941 >> 172.21.15.37:0/812068941 conn(0x559f98ba4d30 msgr2=0x559f98ba3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 -- 172.21.15.37:0/812068941 shutdown_connections 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 -- 172.21.15.37:0/812068941 wait complete. 2024-04-26T01:59:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 -- start start 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.035+0000 7faf1de2f740 1 Processor -- start 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 0x559f98bbbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf141aea90 con 0x559f98bb3a00 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7faf141aff00 con 0x559f98ba9140 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7faf141af160 con 0x559f98ba8820 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf1de2f740 10 client.0 _ll_register_callbacks cb 0x7faf0c09ca30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 0x559f98bbbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf13e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 0x559f98bbbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:32854/0 (socket says 172.21.15.37:32854) 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf19000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf13e00640 1 -- 172.21.15.37:0/2270897748 learned_addr learned my addr 172.21.15.37:0/2270897748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.036+0000 7faf13400640 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf13e00640 1 -- 172.21.15.37:0/2270897748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 msgr2=0x559f98bb92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf13e00640 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf13e00640 1 -- 172.21.15.37:0/2270897748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 msgr2=0x559f98bb99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf13e00640 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf13e00640 1 -- 172.21.15.37:0/2270897748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f98c29310 con 0x559f98ba8820 2024-04-26T01:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf13e00640 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 0x559f98bbbf30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf04005ec0 tx=0x7faf04009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ccc237a692a1ac26 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faefe000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faefe000640 1 -- 172.21.15.37:0/2270897748 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04035040 con 0x559f98ba8820 2024-04-26T01:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faefe000640 1 -- 172.21.15.37:0/2270897748 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7faf04054480 con 0x559f98ba8820 2024-04-26T01:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf0c099490 con 0x559f98ba8820 2024-04-26T01:59:32.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faefe000640 1 -- 172.21.15.37:0/2270897748 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7faf04064050 con 0x559f98ba8820 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf1de2f740 10 client.25282 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.037+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x559f98c2a500 con 0x559f98ba8820 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faefe000640 1 -- 172.21.15.37:0/2270897748 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf0c099490 con 0x559f98ba8820 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faefe000640 1 -- 172.21.15.37:0/2270897748 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7faf04044070 con 0x559f98ba8820 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf1de2f740 10 client.25282 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf1de2f740 10 client.25282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf1de2f740 20 client.25282 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98c2ac30 0x559f98c2d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf1de2f740 4 client.25282 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f98b1c320 con 0x559f98c2ac30 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.038+0000 7faf13400640 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98c2ac30 0x559f98c2d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.039+0000 7faf13400640 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98c2ac30 0x559f98c2d050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d5170410600a2d server_cookie=71c2ce6391ee5032 in_seq=0 out_seq=0 2024-04-26T01:59:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.039+0000 7faefe000640 10 client.25282 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faefe000640 1 -- 172.21.15.37:0/2270897748 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f98b1c320 con 0x559f98c2ac30 2024-04-26T01:59:32.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faefe000640 10 client.25282 handle_command_reply: tid=0 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 client.25282 shutdown 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 msgr2=0x559f98bbbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 0x559f98bbbf30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faf04005ec0 tx=0x7faf04009260 comp rx=0 tx=0).stop 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 shutdown_connections 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x559f98c2ac30 0x559f98c2d050 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559f98bb3a00 0x559f98bb99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x559f98ba9140 0x559f98bb92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faefe000640 0 client.25282 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 --2- 172.21.15.37:0/2270897748 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x559f98ba8820 0x559f98bbbf30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 >> 172.21.15.37:0/2270897748 conn(0x559f98ba4d30 msgr2=0x559f98ba3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 shutdown_connections 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 1 -- 172.21.15.37:0/2270897748 wait complete. 2024-04-26T01:59:32.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.045+0000 7faf1de2f740 20 client.25282 trim_cache size 0 max 16384 2024-04-26T01:59:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:59:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:59:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:59:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:59:32.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.046+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7faef80a3d10 msgr2=0x7faef80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.046+0000 7faf1b000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7faef80a3d10 0x7faef80a61d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x559f98ba9dc0 tx=0x7faf04005a20 comp rx=0 tx=0).stop 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.046+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1419a7e0 msgr2=0x7faf1419ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.046+0000 7faf1b000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1419a7e0 0x7faf1419ac90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009fd0 tx=0x7faf0c00ef10 comp rx=0 tx=0).stop 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 shutdown_connections 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7faef80a3d10 0x7faef80a61d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7faf1419a7e0 0x7faf1419ac90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf14109490 0x7faf141a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 --2- 172.21.15.37:0/4158810395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7faf141043c0 0x7faf141a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 >> 172.21.15.37:0/4158810395 conn(0x7faf140ffc40 msgr2=0x7faf14100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 shutdown_connections 2024-04-26T01:59:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:32.047+0000 7faf1b000640 1 -- 172.21.15.37:0/4158810395 wait complete. 2024-04-26T01:59:32.103 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:31 smithi037 ceph-mon[27906]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T01:59:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 -- 172.21.15.37:0/2424246765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f04108a30 msgr2=0x7f7f04108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/2424246765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f04108a30 0x7f7f04108e10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b0a0 tx=0x7f7efc00b3b0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 -- 172.21.15.37:0/2424246765 shutdown_connections 2024-04-26T01:59:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/2424246765 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 0x7f7f0410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/2424246765 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f04108a30 0x7f7f04108e10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/2424246765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f0406a750 0x7f7f0406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 -- 172.21.15.37:0/2424246765 >> 172.21.15.37:0/2424246765 conn(0x7f7f040788f0 msgr2=0x7f7f04078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 -- 172.21.15.37:0/2424246765 shutdown_connections 2024-04-26T01:59:32.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.165+0000 7f7f0bc00700 1 -- 172.21.15.37:0/2424246765 wait complete. 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.166+0000 7f7f0bc00700 1 Processor -- start 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 -- start start 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0406a750 0x7f7f0419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 0x7f7f0419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f04108a30 0x7f7f0419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04112c00 con 0x7f7f04108a30 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04112a80 con 0x7f7f0406a750 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.167+0000 7f7f0bc00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f04112d80 con 0x7f7f0406b0f0 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 0x7f7f0419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0406a750 0x7f7f0419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 0x7f7f0419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:32880/0 (socket says 172.21.15.37:32880) 2024-04-26T01:59:32.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f03e00700 1 -- 172.21.15.37:0/3085679226 learned_addr learned my addr 172.21.15.37:0/3085679226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09c00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f04108a30 0x7f7f0419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 -- 172.21.15.37:0/3085679226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 msgr2=0x7f7f0419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 0x7f7f0419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 -- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f04108a30 msgr2=0x7f7f0419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f04108a30 0x7f7f0419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 -- 172.21.15.37:0/3085679226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f041a03c0 con 0x7f7f0406a750 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.168+0000 7f7f09200700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0406a750 0x7f7f0419b3b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b070 tx=0x7f7efc004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.169+0000 7f7f01600700 1 -- 172.21.15.37:0/3085679226 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc021070 con 0x7f7f0406a750 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.169+0000 7f7f01600700 1 -- 172.21.15.37:0/3085679226 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7efc007e30 con 0x7f7f0406a750 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.169+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f041a0650 con 0x7f7f0406a750 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.169+0000 7f7f01600700 1 -- 172.21.15.37:0/3085679226 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0046a0 con 0x7f7f0406a750 2024-04-26T01:59:32.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.169+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041a7f30 con 0x7f7f0406a750 2024-04-26T01:59:32.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.172+0000 7f7f01600700 1 -- 172.21.15.37:0/3085679226 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7efc007980 con 0x7f7f0406a750 2024-04-26T01:59:32.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.174+0000 7f7f16630b80 1 Processor -- start 2024-04-26T01:59:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 -- start start 2024-04-26T01:59:32.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f01600700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7ee80a3e30 0x7f7ee80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50368a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503694f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036978c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c5035dcf90 con 0x55c50368a660 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c503583830 con 0x55c503689d40 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f16630b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c5035d8f50 con 0x55c5036954d0 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f01600700 1 -- 172.21.15.37:0/3085679226 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7efc00f020 con 0x7f7f0406a750 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.175+0000 7f7f09c00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7ee80a3e30 0x7f7ee80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50368a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50368a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:45074/0 (socket says 172.21.15.37:45074) 2024-04-26T01:59:32.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 -- 172.21.15.37:0/171487333 learned_addr learned my addr 172.21.15.37:0/171487333 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09c00700 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f03e00700 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503694f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 -- 172.21.15.37:0/171487333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 msgr2=0x55c5036978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 -- 172.21.15.37:0/171487333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 msgr2=0x55c503694f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503694f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 -- 172.21.15.37:0/171487333 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c503579780 con 0x55c503689d40 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09200700 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50368a120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0a60d0 tx=0x7f7efc009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc0be280705b0f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.176+0000 7f7f09c00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7ee80a3e30 0x7f7ee80a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40097a0 tx=0x7f7ef4009b60 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7efa000700 1 -- 172.21.15.37:0/171487333 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc029a10 con 0x55c503689d40 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7efa000700 1 -- 172.21.15.37:0/171487333 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7efc007c70 con 0x55c503689d40 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7efa000700 1 -- 172.21.15.37:0/171487333 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0a1dd0 con 0x55c503689d40 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 -- 172.21.15.37:0/171487333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 msgr2=0x55c50368a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50368a120 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0a60d0 tx=0x7f7efc009c50 comp rx=0 tx=0).stop 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 -- 172.21.15.37:0/171487333 shutdown_connections 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50368a120 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 --2- 172.21.15.37:0/171487333 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503694f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.177+0000 7f7f16630b80 1 -- 172.21.15.37:0/171487333 >> 172.21.15.37:0/171487333 conn(0x55c5035b9050 msgr2=0x55c503686250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.178+0000 7f7f16630b80 1 -- 172.21.15.37:0/171487333 shutdown_connections 2024-04-26T01:59:32.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.178+0000 7f7f16630b80 1 -- 172.21.15.37:0/171487333 wait complete. 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.178+0000 7f7f16630b80 1 -- start start 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.178+0000 7f7f16630b80 1 Processor -- start 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c503689d40 0x55c503694bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503691ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5036954d0 0x55c503692430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c503583830 con 0x55c50368a660 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c503596960 con 0x55c5036954d0 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c503697e00 con 0x55c503689d40 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f16630b80 10 client.0 _ll_register_callbacks cb 0x55c503575820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f03e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503691ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f03e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503691ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60512/0 (socket says 172.21.15.37:60512) 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f03e00700 1 -- 172.21.15.37:0/1558655111 learned_addr learned my addr 172.21.15.37:0/1558655111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.179+0000 7f7f09200700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c503689d40 0x55c503694bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09c00700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5036954d0 0x55c503692430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09200700 1 -- 172.21.15.37:0/1558655111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5036954d0 msgr2=0x55c503692430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09200700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5036954d0 0x55c503692430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09200700 1 -- 172.21.15.37:0/1558655111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 msgr2=0x55c503691ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09200700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503691ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09200700 1 -- 172.21.15.37:0/1558655111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50352a9d0 con 0x55c503689d40 2024-04-26T01:59:32.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f03e00700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503691ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f09200700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c503689d40 0x55c503694bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e390 tx=0x7f7efc0be110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fba7412f54da577 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7ef2a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:59:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1558655111 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00e1d0 con 0x55c503689d40 2024-04-26T01:59:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1558655111 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7efc0074e0 con 0x55c503689d40 2024-04-26T01:59:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.180+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c5035ea780 con 0x55c503689d40 2024-04-26T01:59:32.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1558655111 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7efc09e120 con 0x55c503689d40 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7f16630b80 10 client.25294 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c503573bf0 con 0x55c503689d40 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1558655111 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c5035ea780 con 0x55c503689d40 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1558655111 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7efc011070 con 0x55c503689d40 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7f16630b80 10 client.25294 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.181+0000 7f7f16630b80 10 client.25294 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.182+0000 7f7f16630b80 20 client.25294 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.182+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c5036a47e0 0x55c5036a6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.182+0000 7f7f16630b80 4 client.25294 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.182+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5035e4820 con 0x55c5036a47e0 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.182+0000 7f7f03e00700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c5036a47e0 0x55c5036a6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.183+0000 7f7ef2a00700 10 client.25294 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.183+0000 7f7f03e00700 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c5036a47e0 0x55c5036a6bd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=166213f8289247f2 server_cookie=1c99671cc41c045f in_seq=0 out_seq=0 2024-04-26T01:59:32.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.183+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1558655111 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c5035e4820 con 0x55c5036a47e0 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.183+0000 7f7ef2a00700 10 client.25294 handle_command_reply: tid=0 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 client.25294 shutdown 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c503689d40 msgr2=0x55c503694bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c503689d40 0x55c503694bf0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00e390 tx=0x7f7efc0be110 comp rx=0 tx=0).stop 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 shutdown_connections 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c5036a47e0 0x55c5036a6bd0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c5036954d0 0x55c503692430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c503689d40 0x55c503694bf0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7ef2a00700 0 client.25294 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1558655111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c503691ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 >> 172.21.15.37:0/1558655111 conn(0x55c5035b9050 msgr2=0x55c503685e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 shutdown_connections 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 1 -- 172.21.15.37:0/1558655111 wait complete. 2024-04-26T01:59:32.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.184+0000 7f7f16630b80 20 client.25294 trim_cache size 0 max 16384 2024-04-26T01:59:32.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 Processor -- start 2024-04-26T01:59:32.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 -- start start 2024-04-26T01:59:32.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c5036a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c5036ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ad330 con 0x55c50368a660 2024-04-26T01:59:32.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041afbb0 con 0x55c503689d40 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.207+0000 7f7f16630b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ad9a0 con 0x55c5036954d0 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c5036a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036ac830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:32906/0 (socket says 172.21.15.37:32906) 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 -- 172.21.15.37:0/792903682 learned_addr learned my addr 172.21.15.37:0/792903682 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f03e00700 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c5036ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 -- 172.21.15.37:0/792903682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 msgr2=0x55c5036a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c5036a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 -- 172.21.15.37:0/792903682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 msgr2=0x55c5036ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c5036ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.208+0000 7f7f09c00700 1 -- 172.21.15.37:0/792903682 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c503687800 con 0x55c5036954d0 2024-04-26T01:59:32.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.209+0000 7f7f09c00700 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036ac830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4013070 tx=0x7f7ef40047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5793775846b42983 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.209+0000 7f7ef9600700 1 -- 172.21.15.37:0/792903682 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4004440 con 0x55c5036954d0 2024-04-26T01:59:32.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.209+0000 7f7ef9600700 1 -- 172.21.15.37:0/792903682 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef4011d80 con 0x55c5036954d0 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.209+0000 7f7ef9600700 1 -- 172.21.15.37:0/792903682 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4003c60 con 0x55c5036954d0 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 -- 172.21.15.37:0/792903682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 msgr2=0x55c5036ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036ac830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4013070 tx=0x7f7ef40047c0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 -- 172.21.15.37:0/792903682 shutdown_connections 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c5036954d0 0x55c5036ac830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c5036a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 --2- 172.21.15.37:0/792903682 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c50368a660 0x55c5036ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 -- 172.21.15.37:0/792903682 >> 172.21.15.37:0/792903682 conn(0x55c5035b9050 msgr2=0x55c503691eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 -- 172.21.15.37:0/792903682 shutdown_connections 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 -- 172.21.15.37:0/792903682 wait complete. 2024-04-26T01:59:32.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.210+0000 7f7f16630b80 1 -- start start 2024-04-26T01:59:32.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 Processor -- start 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50369a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50368a660 0x55c50369a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 0x55c50370ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ae070 con 0x55c5036954d0 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041af4e0 con 0x55c503689d40 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7f041ae740 con 0x55c50368a660 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f16630b80 10 client.0 _ll_register_callbacks cb 0x55c503575820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f09c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 0x55c50370ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f09c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 0x55c50370ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60536/0 (socket says 172.21.15.37:60536) 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f09c00700 1 -- 172.21.15.37:0/1183660665 learned_addr learned my addr 172.21.15.37:0/1183660665 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:32.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f03e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50368a660 0x55c50369a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.211+0000 7f7f09200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50369a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09200700 1 -- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50368a660 msgr2=0x55c50369a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09200700 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50368a660 0x55c50369a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09200700 1 -- 172.21.15.37:0/1183660665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 msgr2=0x55c50370ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09200700 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 0x55c50370ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09200700 1 -- 172.21.15.37:0/1183660665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c50370b160 con 0x55c503689d40 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09c00700 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 0x55c50370ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7f09200700 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50369a2a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7efc09d720 tx=0x7f7efc009960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb1a5a6611e42730 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:32.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.212+0000 7f7ef2a00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1183660665 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc09d7f0 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1183660665 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7efc0a42d0 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ef4011d80 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1183660665 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7efc00e1d0 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 10 client.25256 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c50370b160 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1183660665 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ef4011d80 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1183660665 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7efc011040 con 0x55c503689d40 2024-04-26T01:59:32.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 10 client.25256 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 10 client.25256 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 20 client.25256 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c50370c760 0x55c50370eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.213+0000 7f7f16630b80 4 client.25256 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "38b727e3", "release": true, "await": true}] 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.214+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "set_id": "38b727e3", "release": true, "await": true}) -- 0x55c50345d350 con 0x55c50370c760 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.214+0000 7f7f03e00700 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c50370c760 0x55c50370eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.214+0000 7f7f03e00700 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c50370c760 0x55c50370eb50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af1333026ea9c6ff server_cookie=107e8dfc073d516a in_seq=0 out_seq=0 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.214+0000 7f7ef2a00700 10 client.25256 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7ef2a00700 1 -- 172.21.15.37:0/1183660665 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55c50345d350 con 0x55c50370c760 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7ef2a00700 10 client.25256 handle_command_reply: tid=0 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 client.25256 shutdown 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 msgr2=0x55c50369a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50369a2a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7efc09d720 tx=0x7f7efc009960 comp rx=0 tx=0).stop 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 shutdown_connections 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c50370c760 0x55c50370eb50 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5036954d0 0x55c50370ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c503689d40 0x55c50369a2a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 --2- 172.21.15.37:0/1183660665 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c50368a660 0x55c50369a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7ef2a00700 0 client.25256 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 >> 172.21.15.37:0/1183660665 conn(0x55c5035b9050 msgr2=0x55c503693720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 shutdown_connections 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.215+0000 7f7f16630b80 1 -- 172.21.15.37:0/1183660665 wait complete. 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f16630b80 20 client.25256 trim_cache size 0 max 16384 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 4, 2024-04-26T01:59:32.216 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "38b727e3": { 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4, 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 85.6, 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T01:59:32.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7ee80a3e30 msgr2=0x7f7ee80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7ee80a3e30 0x7f7ee80a62f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40097a0 tx=0x7f7ef4009b60 comp rx=0 tx=0).stop 2024-04-26T01:59:32.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0406a750 msgr2=0x7f7f0419b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0406a750 0x7f7f0419b3b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00b070 tx=0x7f7efc004d00 comp rx=0 tx=0).stop 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 shutdown_connections 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f04108a30 0x7f7f0419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7ee80a3e30 0x7f7ee80a62f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7f0406a750 0x7f7f0419b3b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 --2- 172.21.15.37:0/3085679226 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7f0406b0f0 0x7f7f0419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.216+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 >> 172.21.15.37:0/3085679226 conn(0x7f7f040788f0 msgr2=0x7f7f0410df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.217+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 shutdown_connections 2024-04-26T01:59:32.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T01:59:32.217+0000 7f7f0bc00700 1 -- 172.21.15.37:0/3085679226 wait complete. 2024-04-26T01:59:32.229 INFO:tasks.workunit.client.0.smithi037.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-26T01:59:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:31 smithi045 ceph-mon[32520]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T01:59:32.374 INFO:tasks.workunit.client.0.smithi037.stdout: Iozone: Performance Test of File I/O 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Version $Revision: 3.494 $ 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Compiled for 64 bit mode. 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Build: linux-AMD64 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T01:59:32.375 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Run began: Fri Apr 26 01:59:32 2024 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Include close in write timing 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Include fsync in write timing 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: File size set to 1048576 kB 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Record Size 16 kB 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Output is in kBytes/sec 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Processor cache line size set to 32 bytes. 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: File stride size set to 17 * record size. 2024-04-26T01:59:32.376 INFO:tasks.workunit.client.0.smithi037.stdout: Throughput test with 1 process 2024-04-26T01:59:32.377 INFO:tasks.workunit.client.0.smithi037.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-04-26T01:59:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:31 smithi071 ceph-mon[32578]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-04-26T01:59:33.131 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '38b727e3', seconds elapsed: 41.5 2024-04-26T01:59:34.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:33 smithi037 ceph-mon[27906]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-04-26T01:59:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:33 smithi045 ceph-mon[32520]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-04-26T01:59:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:33 smithi071 ceph-mon[32578]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 5.1 KiB/s wr, 5 op/s 2024-04-26T01:59:36.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:35 smithi037 ceph-mon[27906]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-04-26T01:59:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:35 smithi045 ceph-mon[32520]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-04-26T01:59:36.286 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:36.286 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 initial writers = 268393.97 kB/sec 2024-04-26T01:59:36.286 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 initial writers = 268341.48 kB/sec 2024-04-26T01:59:36.286 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 268393.97 kB/sec 2024-04-26T01:59:36.286 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 268393.97 kB/sec 2024-04-26T01:59:36.287 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 268393.97 kB/sec 2024-04-26T01:59:36.287 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T01:59:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:35 smithi071 ceph-mon[32578]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-04-26T01:59:38.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:37 smithi037 ceph-mon[27906]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-04-26T01:59:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:37 smithi045 ceph-mon[32520]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-04-26T01:59:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:37 smithi071 ceph-mon[32578]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 4.7 KiB/s wr, 3 op/s 2024-04-26T01:59:40.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:39 smithi037 ceph-mon[27906]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T01:59:40.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:39 smithi045 ceph-mon[32520]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T01:59:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:39 smithi071 ceph-mon[32578]: pgmap v54: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T01:59:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:41 smithi037 ceph-mon[27906]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T01:59:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:41 smithi045 ceph-mon[32520]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T01:59:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:41 smithi071 ceph-mon[32578]: pgmap v55: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 85 MiB/s wr, 25 op/s 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 rewriters = 338376.62 kB/sec 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 rewriters = 338301.49 kB/sec 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 338376.62 kB/sec 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 338376.62 kB/sec 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 338376.62 kB/sec 2024-04-26T01:59:43.285 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T01:59:44.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:43 smithi037 ceph-mon[27906]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 142 MiB/s wr, 40 op/s 2024-04-26T01:59:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:43 smithi045 ceph-mon[32520]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 142 MiB/s wr, 40 op/s 2024-04-26T01:59:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:43 smithi071 ceph-mon[32578]: pgmap v56: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 142 MiB/s wr, 40 op/s 2024-04-26T01:59:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 -- 172.21.15.37:0/680834065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 msgr2=0x7f526c10ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 --2- 172.21.15.37:0/680834065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c10ed50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5264009ca0 tx=0x7f526402a8b0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 -- 172.21.15.37:0/680834065 shutdown_connections 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 --2- 172.21.15.37:0/680834065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c10ed50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 --2- 172.21.15.37:0/680834065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c1002b0 0x7f526c100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 --2- 172.21.15.37:0/680834065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f526c106ea0 0x7f526c107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 -- 172.21.15.37:0/680834065 >> 172.21.15.37:0/680834065 conn(0x7f526c077210 msgr2=0x7f526c0fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 -- 172.21.15.37:0/680834065 shutdown_connections 2024-04-26T01:59:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.197+0000 7f5271400640 1 -- 172.21.15.37:0/680834065 wait complete. 2024-04-26T01:59:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.198+0000 7f5271400640 1 Processor -- start 2024-04-26T01:59:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.198+0000 7f5271400640 1 -- start start 2024-04-26T01:59:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5271400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f526c1002b0 0x7f526c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5271400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5271400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 0x7f526c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5271400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f526c112230 con 0x7f526c106ea0 2024-04-26T01:59:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5271400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1120b0 con 0x7f526c100c70 2024-04-26T01:59:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5271400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1123b0 con 0x7f526c1002b0 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5270a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 0x7f526c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f526be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f526c1002b0 0x7f526c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5270a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 0x7f526c19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33312/0 (socket says 172.21.15.37:33312) 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f5270a00640 1 -- 172.21.15.37:0/2770412552 learned_addr learned my addr 172.21.15.37:0/2770412552 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.199+0000 7f526b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f526b400640 1 -- 172.21.15.37:0/2770412552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f526c1002b0 msgr2=0x7f526c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f526b400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f526c1002b0 0x7f526c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f526b400640 1 -- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 msgr2=0x7f526c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f526b400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 0x7f526c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f526b400640 1 -- 172.21.15.37:0/2770412552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f526c19d680 con 0x7f526c100c70 2024-04-26T01:59:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f5270a00640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 0x7f526c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.200+0000 7f526b400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c198bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f525800d6d0 tx=0x7f525800db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.201+0000 7f5268c00640 1 -- 172.21.15.37:0/2770412552 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258017070 con 0x7f526c100c70 2024-04-26T01:59:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.201+0000 7f5268c00640 1 -- 172.21.15.37:0/2770412552 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5258009820 con 0x7f526c100c70 2024-04-26T01:59:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.201+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f526c19d970 con 0x7f526c100c70 2024-04-26T01:59:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.201+0000 7f5268c00640 1 -- 172.21.15.37:0/2770412552 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5258004c40 con 0x7f526c100c70 2024-04-26T01:59:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.201+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f526c1a5200 con 0x7f526c100c70 2024-04-26T01:59:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.204+0000 7f52759be740 1 Processor -- start 2024-04-26T01:59:45.203 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.204+0000 7f52759be740 1 -- start start 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f52759be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 0x555fbec1cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f52759be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 0x555fbec2ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f52759be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec2e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f52759be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fbeb82900 con 0x555fbec2bfa0 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f52759be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555fbeb28ef0 con 0x555fbec1c870 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f52759be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fbeb7e8c0 con 0x555fbec1d190 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f526be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 0x555fbec2ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f5270a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 0x555fbec1cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f526be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 0x555fbec2ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54770/0 (socket says 172.21.15.37:54770) 2024-04-26T01:59:45.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f5270a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 0x555fbec1cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56312/0 (socket says 172.21.15.37:56312) 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.205+0000 7f526be00640 1 -- 172.21.15.37:0/2343452608 learned_addr learned my addr 172.21.15.37:0/2343452608 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f526be00640 1 -- 172.21.15.37:0/2343452608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 msgr2=0x555fbec1cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f526be00640 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 0x555fbec1cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f5268c00640 1 -- 172.21.15.37:0/2770412552 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5258005220 con 0x7f526c100c70 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f526be00640 1 -- 172.21.15.37:0/2343452608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 msgr2=0x555fbec2e390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f526be00640 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec2e390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f526be00640 1 -- 172.21.15.37:0/2343452608 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fbe895210 con 0x555fbec1d190 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f5270a00640 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 0x555fbec1cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:45.206 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f526be00640 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 0x555fbec2ba60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f525c009cf0 tx=0x7f525c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b2bc53d209794ee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f5261200640 1 -- 172.21.15.37:0/2343452608 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c004230 con 0x555fbec1d190 2024-04-26T01:59:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f5261200640 1 -- 172.21.15.37:0/2343452608 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f525c0043d0 con 0x555fbec1d190 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.206+0000 7f5261200640 1 -- 172.21.15.37:0/2343452608 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525c019880 con 0x555fbec1d190 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 -- 172.21.15.37:0/2343452608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 msgr2=0x555fbec2ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 0x555fbec2ba60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f525c009cf0 tx=0x7f525c011810 comp rx=0 tx=0).stop 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 -- 172.21.15.37:0/2343452608 shutdown_connections 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec2e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1d190 0x555fbec2ba60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 --2- 172.21.15.37:0/2343452608 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1c870 0x555fbec1cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 -- 172.21.15.37:0/2343452608 >> 172.21.15.37:0/2343452608 conn(0x555fbec18ec0 msgr2=0x555fbec192a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 -- 172.21.15.37:0/2343452608 shutdown_connections 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f52759be740 1 -- 172.21.15.37:0/2343452608 wait complete. 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f5268c00640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f52500a3c10 0x7f52500a60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.207+0000 7f5268c00640 1 -- 172.21.15.37:0/2770412552 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f52580d0270 con 0x7f526c100c70 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f526be00640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f52500a3c10 0x7f52500a60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 -- start start 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 Processor -- start 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec28fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec294f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f526be00640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f52500a3c10 0x7f52500a60d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f525c009cf0 tx=0x7f525c000bc0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec29a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fbec2eb70 con 0x555fbec1c870 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555fbeb3c310 con 0x555fbec1d190 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.208+0000 7f52759be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fbec2ecf0 con 0x555fbec2bfa0 2024-04-26T01:59:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f52759be740 10 client.0 _ll_register_callbacks cb 0x555fbeb0acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec28fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec28fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33314/0 (socket says 172.21.15.37:33314) 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526be00640 1 -- 172.21.15.37:0/1251018898 learned_addr learned my addr 172.21.15.37:0/1251018898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526b400640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec294f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f5270a00640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec29a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526b400640 1 -- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 msgr2=0x555fbec29a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526b400640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec29a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526b400640 1 -- 172.21.15.37:0/1251018898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 msgr2=0x555fbec28fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526b400640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec28fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526b400640 1 -- 172.21.15.37:0/1251018898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fbe961620 con 0x555fbec1d190 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.209+0000 7f526be00640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec28fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f526b400640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec294f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52580cd990 tx=0x7f525809c3d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5561a066a34e2f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f5255600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:59:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f5255600640 1 -- 172.21.15.37:0/1251018898 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580d0070 con 0x555fbec1d190 2024-04-26T01:59:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f525c0043d0 con 0x555fbec1d190 2024-04-26T01:59:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f5255600640 1 -- 172.21.15.37:0/1251018898 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f52580099d0 con 0x555fbec1d190 2024-04-26T01:59:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f5255600640 1 -- 172.21.15.37:0/1251018898 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f5258020040 con 0x555fbec1d190 2024-04-26T01:59:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f52759be740 10 client.25268 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555fbec3af50 con 0x555fbec1d190 2024-04-26T01:59:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.210+0000 7f5255600640 1 -- 172.21.15.37:0/1251018898 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f525c0043d0 con 0x555fbec1d190 2024-04-26T01:59:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f5255600640 1 -- 172.21.15.37:0/1251018898 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f52580a0120 con 0x555fbec1d190 2024-04-26T01:59:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f52759be740 10 client.25268 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f52759be740 10 client.25268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f52759be740 20 client.25268 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f52759be740 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbec3c3c0 0x555fbec3e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f52759be740 4 client.25268 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fbeb968b0 con 0x555fbec3c3c0 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.211+0000 7f526be00640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbec3c3c0 0x555fbec3e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.212+0000 7f526be00640 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbec3c3c0 0x555fbec3e7b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c270e72ffb1618c server_cookie=fea216b30c7c41b7 in_seq=0 out_seq=0 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.212+0000 7f5255600640 10 client.25268 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f5255600640 1 -- 172.21.15.37:0/1251018898 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fbeb968b0 con 0x555fbec3c3c0 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f5255600640 10 client.25268 handle_command_reply: tid=0 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 client.25268 shutdown 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 msgr2=0x555fbec294f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec294f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f52580cd990 tx=0x7f525809c3d0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 shutdown_connections 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbec3c3c0 0x555fbec3e7b0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec29a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f5255600640 0 client.25268 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec294f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 --2- 172.21.15.37:0/1251018898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec28fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 >> 172.21.15.37:0/1251018898 conn(0x555fbec18ec0 msgr2=0x555fbec1ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 shutdown_connections 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 1 -- 172.21.15.37:0/1251018898 wait complete. 2024-04-26T01:59:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.213+0000 7f52759be740 20 client.25268 trim_cache size 0 max 16384 2024-04-26T01:59:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 Processor -- start 2024-04-26T01:59:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 -- start start 2024-04-26T01:59:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1c870 0x555fbec37a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec44070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec445b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1a96f0 con 0x555fbec2bfa0 2024-04-26T01:59:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1abf70 con 0x555fbec1d190 2024-04-26T01:59:45.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f52759be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1a9d60 con 0x555fbec1c870 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f526b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec44070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.230+0000 7f526be00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1c870 0x555fbec37a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec44070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56322/0 (socket says 172.21.15.37:56322) 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 -- 172.21.15.37:0/2065306914 learned_addr learned my addr 172.21.15.37:0/2065306914 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f5270a00640 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 -- 172.21.15.37:0/2065306914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1c870 msgr2=0x555fbec37a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1c870 0x555fbec37a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 -- 172.21.15.37:0/2065306914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 msgr2=0x555fbec445b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec445b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 -- 172.21.15.37:0/2065306914 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fbec3ede0 con 0x555fbec1d190 2024-04-26T01:59:45.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.231+0000 7f526b400640 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec44070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f525809a540 tx=0x7f52580d04f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a033c3d2cc33e9d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.232+0000 7f5257e00640 1 -- 172.21.15.37:0/2065306914 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f525809a650 con 0x555fbec1d190 2024-04-26T01:59:45.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.232+0000 7f5257e00640 1 -- 172.21.15.37:0/2065306914 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52580a4a90 con 0x555fbec1d190 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.232+0000 7f5257e00640 1 -- 172.21.15.37:0/2065306914 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52580a0120 con 0x555fbec1d190 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 -- 172.21.15.37:0/2065306914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 msgr2=0x555fbec44070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec44070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f525809a540 tx=0x7f52580d04f0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 -- 172.21.15.37:0/2065306914 shutdown_connections 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec2bfa0 0x555fbec445b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec44070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 --2- 172.21.15.37:0/2065306914 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec1c870 0x555fbec37a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 -- 172.21.15.37:0/2065306914 >> 172.21.15.37:0/2065306914 conn(0x555fbec18ec0 msgr2=0x555fbec29800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 -- 172.21.15.37:0/2065306914 shutdown_connections 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 -- 172.21.15.37:0/2065306914 wait complete. 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 -- start start 2024-04-26T01:59:45.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.233+0000 7f52759be740 1 Processor -- start 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec342a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec31b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1aa430 con 0x555fbec1c870 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1ab8a0 con 0x555fbec1d190 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f526c1aab00 con 0x555fbec2bfa0 2024-04-26T01:59:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f52759be740 10 client.0 _ll_register_callbacks cb 0x7f525809bbf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f5270a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec31b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f5270a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec31b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:54798/0 (socket says 172.21.15.37:54798) 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f5270a00640 1 -- 172.21.15.37:0/3593575812 learned_addr learned my addr 172.21.15.37:0/3593575812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f526be00640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.234+0000 7f526b400640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5270a00640 1 -- 172.21.15.37:0/3593575812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 msgr2=0x555fbec315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5270a00640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5270a00640 1 -- 172.21.15.37:0/3593575812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 msgr2=0x555fbec342a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5270a00640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec342a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5270a00640 1 -- 172.21.15.37:0/3593575812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fbec32260 con 0x555fbec2bfa0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5270a00640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec31b20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5264006010 tx=0x7f5264003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a099f2bb33b88cfb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5255600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5255600640 1 -- 172.21.15.37:0/3593575812 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004240 con 0x555fbec2bfa0 2024-04-26T01:59:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52580a4a90 con 0x555fbec2bfa0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.235+0000 7f5255600640 1 -- 172.21.15.37:0/3593575812 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f52640043e0 con 0x555fbec2bfa0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f5255600640 1 -- 172.21.15.37:0/3593575812 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f52640045e0 con 0x555fbec2bfa0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 10 client.25318 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555fbeca2880 con 0x555fbec2bfa0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f5255600640 1 -- 172.21.15.37:0/3593575812 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52580a4a90 con 0x555fbec2bfa0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f5255600640 1 -- 172.21.15.37:0/3593575812 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f5264031440 con 0x555fbec2bfa0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 10 client.25318 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 10 client.25318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 20 client.25318 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbeca2fe0 0x555fbeca5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 4 client.25318 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fbeb90320 con 0x555fbeca2fe0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.236+0000 7f526be00640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbeca2fe0 0x555fbeca5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.237+0000 7f526be00640 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbeca2fe0 0x555fbeca5400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2908e20b683fb2d server_cookie=9d4f836e293ff659 in_seq=0 out_seq=0 2024-04-26T01:59:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.237+0000 7f5255600640 10 client.25318 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f5255600640 1 -- 172.21.15.37:0/3593575812 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fbeb90320 con 0x555fbeca2fe0 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f5255600640 10 client.25318 handle_command_reply: tid=0 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 client.25318 shutdown 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 msgr2=0x555fbec31b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec31b20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5264006010 tx=0x7f5264003ba0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 shutdown_connections 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fbeca2fe0 0x555fbeca5400 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fbec2bfa0 0x555fbec31b20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fbec1d190 0x555fbec315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 --2- 172.21.15.37:0/3593575812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fbec1c870 0x555fbec342a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f5255600640 0 client.25318 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 >> 172.21.15.37:0/3593575812 conn(0x555fbec18ec0 msgr2=0x555fbec17790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 shutdown_connections 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.247+0000 7f52759be740 1 -- 172.21.15.37:0/3593575812 wait complete. 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.248+0000 7f52759be740 20 client.25318 trim_cache size 0 max 16384 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:59:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:59:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:59:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:59:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.248+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f52500a3c10 msgr2=0x7f52500a60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.248+0000 7f5271400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f52500a3c10 0x7f52500a60d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f525c009cf0 tx=0x7f525c000bc0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.248+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 msgr2=0x7f526c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.248+0000 7f5271400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c198bd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f525800d6d0 tx=0x7f525800db90 comp rx=0 tx=0).stop 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 shutdown_connections 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f52500a3c10 0x7f52500a60d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f526c106ea0 0x7f526c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f526c100c70 0x7f526c198bd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 --2- 172.21.15.37:0/2770412552 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f526c1002b0 0x7f526c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 >> 172.21.15.37:0/2770412552 conn(0x7f526c077210 msgr2=0x7f526c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 shutdown_connections 2024-04-26T01:59:45.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:45.249+0000 7f5271400640 1 -- 172.21.15.37:0/2770412552 wait complete. 2024-04-26T01:59:45.729 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:45.729 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 readers = 3509512.50 kB/sec 2024-04-26T01:59:45.729 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 readers = 3499838.21 kB/sec 2024-04-26T01:59:45.729 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 3509512.50 kB/sec 2024-04-26T01:59:45.729 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 3509512.50 kB/sec 2024-04-26T01:59:45.730 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 3509512.50 kB/sec 2024-04-26T01:59:45.730 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T01:59:46.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:45 smithi037 ceph-mon[27906]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 148 MiB/s wr, 39 op/s 2024-04-26T01:59:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:45 smithi045 ceph-mon[32520]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 148 MiB/s wr, 39 op/s 2024-04-26T01:59:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:45 smithi071 ceph-mon[32578]: pgmap v57: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 148 MiB/s wr, 39 op/s 2024-04-26T01:59:48.205 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:48.205 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 re-readers = 2625436.50 kB/sec 2024-04-26T01:59:48.205 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 re-readers = 2620129.40 kB/sec 2024-04-26T01:59:48.206 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 2625436.50 kB/sec 2024-04-26T01:59:48.206 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 2625436.50 kB/sec 2024-04-26T01:59:48.206 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 2625436.50 kB/sec 2024-04-26T01:59:48.206 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T01:59:48.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:47 smithi037 ceph-mon[27906]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 148 MiB/s wr, 39 op/s 2024-04-26T01:59:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:47 smithi045 ceph-mon[32520]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 148 MiB/s wr, 39 op/s 2024-04-26T01:59:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:47 smithi071 ceph-mon[32578]: pgmap v58: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 148 MiB/s wr, 39 op/s 2024-04-26T01:59:50.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:49 smithi045 ceph-mon[32520]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 49 op/s 2024-04-26T01:59:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:49 smithi071 ceph-mon[32578]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 49 op/s 2024-04-26T01:59:50.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:49 smithi037 ceph-mon[27906]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 171 MiB/s wr, 49 op/s 2024-04-26T01:59:52.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:51 smithi045 ceph-mon[32520]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T01:59:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:51 smithi071 ceph-mon[32578]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T01:59:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:51 smithi037 ceph-mon[27906]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 85 MiB/s wr, 26 op/s 2024-04-26T01:59:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:53 smithi045 ceph-mon[32520]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T01:59:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:53 smithi071 ceph-mon[32578]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T01:59:54.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:53 smithi037 ceph-mon[27906]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 85 MiB/s wr, 27 op/s 2024-04-26T01:59:54.746 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:54.746 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:54.746 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:54.746 INFO:tasks.workunit.client.0.smithi037.stdout:iozone test complete. 2024-04-26T01:59:54.752 INFO:tasks.workunit.client.0.smithi037.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-26T01:59:54.844 INFO:tasks.workunit.client.0.smithi037.stdout: Iozone: Performance Test of File I/O 2024-04-26T01:59:54.844 INFO:tasks.workunit.client.0.smithi037.stdout: Version $Revision: 3.494 $ 2024-04-26T01:59:54.844 INFO:tasks.workunit.client.0.smithi037.stdout: Compiled for 64 bit mode. 2024-04-26T01:59:54.844 INFO:tasks.workunit.client.0.smithi037.stdout: Build: linux-AMD64 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: Run began: Fri Apr 26 01:59:54 2024 2024-04-26T01:59:54.845 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Include close in write timing 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Include fsync in write timing 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: File size set to 1048576 kB 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Record Size 1024 kB 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Output is in kBytes/sec 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Processor cache line size set to 32 bytes. 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: File stride size set to 17 * record size. 2024-04-26T01:59:54.846 INFO:tasks.workunit.client.0.smithi037.stdout: Throughput test with 1 process 2024-04-26T01:59:54.847 INFO:tasks.workunit.client.0.smithi037.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-04-26T01:59:56.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:55 smithi045 ceph-mon[32520]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 28 MiB/s wr, 11 op/s 2024-04-26T01:59:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:55 smithi071 ceph-mon[32578]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 28 MiB/s wr, 11 op/s 2024-04-26T01:59:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:55 smithi037 ceph-mon[27906]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 28 MiB/s wr, 11 op/s 2024-04-26T01:59:58.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 01:59:57 smithi037 ceph-mon[27906]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 10 op/s 2024-04-26T01:59:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 01:59:58 smithi045 ceph-mon[32520]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 10 op/s 2024-04-26T01:59:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 01:59:58 smithi071 ceph-mon[32578]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 22 MiB/s wr, 10 op/s 2024-04-26T01:59:58.431 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T01:59:58.434 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 initial writers = 295518.53 kB/sec 2024-04-26T01:59:58.435 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 initial writers = 295416.21 kB/sec 2024-04-26T01:59:58.435 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 295518.53 kB/sec 2024-04-26T01:59:58.435 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 295518.53 kB/sec 2024-04-26T01:59:58.435 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 295518.53 kB/sec 2024-04-26T01:59:58.435 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T01:59:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 -- 172.21.15.37:0/1848186311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20241072a0 msgr2=0x7f202410ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 --2- 172.21.15.37:0/1848186311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20241072a0 0x7f202410ee70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f201c009ca0 tx=0x7f201c02a8c0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 -- 172.21.15.37:0/1848186311 shutdown_connections 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 --2- 172.21.15.37:0/1848186311 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f20241072a0 0x7f202410ee70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 --2- 172.21.15.37:0/1848186311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2024102b80 0x7f2024106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 --2- 172.21.15.37:0/1848186311 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241021d0 0x7f20241025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 -- 172.21.15.37:0/1848186311 >> 172.21.15.37:0/1848186311 conn(0x7f20240fda10 msgr2=0x7f20240ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 -- 172.21.15.37:0/1848186311 shutdown_connections 2024-04-26T01:59:58.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.448+0000 7f202a200640 1 -- 172.21.15.37:0/1848186311 wait complete. 2024-04-26T01:59:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.450+0000 7f202a200640 1 Processor -- start 2024-04-26T01:59:58.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.450+0000 7f202a200640 1 -- start start 2024-04-26T01:59:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f202a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 0x7f2024198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f202a200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241072a0 0x7f20241989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f202a200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202419cd20 0x7f202419d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f202a200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2024112240 con 0x7f20241021d0 2024-04-26T01:59:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f202a200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20241120c0 con 0x7f20241072a0 2024-04-26T01:59:58.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f202a200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20241123c0 con 0x7f202419cd20 2024-04-26T01:59:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.451+0000 7f2023400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 0x7f2024198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2023400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 0x7f2024198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34092/0 (socket says 172.21.15.37:34092) 2024-04-26T01:59:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2023400640 1 -- 172.21.15.37:0/523985073 learned_addr learned my addr 172.21.15.37:0/523985073 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2023e00640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202419cd20 0x7f202419d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2022a00640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241072a0 0x7f20241989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2023400640 1 -- 172.21.15.37:0/523985073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202419cd20 msgr2=0x7f202419d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2023400640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202419cd20 0x7f202419d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.452+0000 7f2023400640 1 -- 172.21.15.37:0/523985073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241072a0 msgr2=0x7f20241989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.453+0000 7f2023400640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241072a0 0x7f20241989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.453+0000 7f2023400640 1 -- 172.21.15.37:0/523985073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202419d810 con 0x7f20241021d0 2024-04-26T01:59:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.453+0000 7f2023e00640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202419cd20 0x7f202419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.453+0000 7f2022a00640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241072a0 0x7f20241989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.453+0000 7f2023400640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 0x7f2024198460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f200c00e9f0 tx=0x7f200c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.454+0000 7f201be00640 1 -- 172.21.15.37:0/523985073 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c00cd80 con 0x7f20241021d0 2024-04-26T01:59:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.454+0000 7f201be00640 1 -- 172.21.15.37:0/523985073 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f200c0108e0 con 0x7f20241021d0 2024-04-26T01:59:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.454+0000 7f201be00640 1 -- 172.21.15.37:0/523985073 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c018980 con 0x7f20241021d0 2024-04-26T01:59:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.454+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f202419db00 con 0x7f20241021d0 2024-04-26T01:59:58.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.454+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20241a5210 con 0x7f20241021d0 2024-04-26T01:59:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.462+0000 7f202cfbe740 1 Processor -- start 2024-04-26T01:59:58.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.462+0000 7f202cfbe740 1 -- start start 2024-04-26T01:59:58.466 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.463+0000 7f201be00640 1 -- 172.21.15.37:0/523985073 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f200c010430 con 0x7f20241021d0 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.463+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059cec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.463+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059d94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.463+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.463+0000 7f202cfbe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556705934900 con 0x5567059d9a00 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.464+0000 7f202cfbe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5567058daef0 con 0x5567059ce820 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.464+0000 7f202cfbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5567059308c0 con 0x5567059cf140 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.464+0000 7f2023e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.464+0000 7f2023e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059cec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56732/0 (socket says 172.21.15.37:56732) 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.464+0000 7f2023e00640 1 -- 172.21.15.37:0/2811955368 learned_addr learned my addr 172.21.15.37:0/2811955368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.464+0000 7f2022a00640 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2023e00640 1 -- 172.21.15.37:0/2811955368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 msgr2=0x5567059d94c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2023e00640 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059d94c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2023e00640 1 -- 172.21.15.37:0/2811955368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 msgr2=0x5567059dbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2023e00640 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2023e00640 1 -- 172.21.15.37:0/2811955368 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556705647210 con 0x5567059ce820 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2022a00640 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.465+0000 7f2023e00640 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059cec00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f201c02ade0 tx=0x7f201c0098b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd72f5a096f0bd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.466+0000 7f2013e00640 1 -- 172.21.15.37:0/2811955368 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c003b70 con 0x5567059ce820 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.466+0000 7f2013e00640 1 -- 172.21.15.37:0/2811955368 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c036030 con 0x5567059ce820 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.466+0000 7f2013e00640 1 -- 172.21.15.37:0/2811955368 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c0308b0 con 0x5567059ce820 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.466+0000 7f201be00640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f20080a3d90 0x7f20080a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.467+0000 7f2023400640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f20080a3d90 0x7f20080a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.466+0000 7f201be00640 1 -- 172.21.15.37:0/523985073 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f200c0d0af0 con 0x7f20241021d0 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.468+0000 7f202cfbe740 1 -- 172.21.15.37:0/2811955368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 msgr2=0x5567059cec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.469+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059cec00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f201c02ade0 tx=0x7f201c0098b0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 -- 172.21.15.37:0/2811955368 shutdown_connections 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059d94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2811955368 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059cec00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 -- 172.21.15.37:0/2811955368 >> 172.21.15.37:0/2811955368 conn(0x5567059cad30 msgr2=0x5567059cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 -- 172.21.15.37:0/2811955368 shutdown_connections 2024-04-26T01:59:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.470+0000 7f202cfbe740 1 -- 172.21.15.37:0/2811955368 wait complete. 2024-04-26T01:59:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.471+0000 7f2023400640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f20080a3d90 0x7f20080a6250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5567059cfdc0 tx=0x7f200c00cf20 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.471+0000 7f202cfbe740 1 -- start start 2024-04-26T01:59:58.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.472+0000 7f202cfbe740 1 Processor -- start 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059d6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059d9a00 0x5567059d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5567059dc450 con 0x5567059cf140 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5567058ee310 con 0x5567059ce820 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5567059dc5d0 con 0x5567059d9a00 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.473+0000 7f202cfbe740 10 client.0 _ll_register_callbacks cb 0x5567058bccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.474+0000 7f2023e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.473 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.474+0000 7f2023e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059d6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56738/0 (socket says 172.21.15.37:56738) 2024-04-26T01:59:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.474+0000 7f2023e00640 1 -- 172.21.15.37:0/2796155341 learned_addr learned my addr 172.21.15.37:0/2796155341 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.474+0000 7f2023e00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059d9a00 0x5567059d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.475+0000 7f2022a00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059d6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.476+0000 7f2022a00640 1 -- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059d9a00 msgr2=0x5567059d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.476+0000 7f2022a00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059d9a00 0x5567059d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.476+0000 7f2022a00640 1 -- 172.21.15.37:0/2796155341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 msgr2=0x5567059d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.476+0000 7f2022a00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059d6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.476+0000 7f2022a00640 1 -- 172.21.15.37:0/2796155341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556705713620 con 0x5567059cf140 2024-04-26T01:59:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.476+0000 7f2023e00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T01:59:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.477+0000 7f2022a00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059d6f80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5567059cbec0 tx=0x7f2014011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd175c7b1d16bd5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.477+0000 7f2010c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T01:59:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.477+0000 7f2010c00640 1 -- 172.21.15.37:0/2796155341 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201400fe40 con 0x5567059cf140 2024-04-26T01:59:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.477+0000 7f2010c00640 1 -- 172.21.15.37:0/2796155341 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f2014005020 con 0x5567059cf140 2024-04-26T01:59:58.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.478+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c036030 con 0x5567059cf140 2024-04-26T01:59:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.478+0000 7f2010c00640 1 -- 172.21.15.37:0/2796155341 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f2014005220 con 0x5567059cf140 2024-04-26T01:59:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.479+0000 7f202cfbe740 10 client.15186 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:58.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.479+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5567059e87d0 con 0x5567059cf140 2024-04-26T01:59:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f2010c00640 1 -- 172.21.15.37:0/2796155341 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c036030 con 0x5567059cf140 2024-04-26T01:59:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f2010c00640 1 -- 172.21.15.37:0/2796155341 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f2014018070 con 0x5567059cf140 2024-04-26T01:59:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f202cfbe740 10 client.15186 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f202cfbe740 10 client.15186 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:58.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f202cfbe740 20 client.15186 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5567059e9d80 0x5567059ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f202cfbe740 4 client.15186 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T01:59:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567059488b0 con 0x5567059e9d80 2024-04-26T01:59:58.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.480+0000 7f2023e00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5567059e9d80 0x5567059ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.482+0000 7f2023e00640 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5567059e9d80 0x5567059ec200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4b350f0fe437f35 server_cookie=f212624c61a15a0e in_seq=0 out_seq=0 2024-04-26T01:59:58.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.482+0000 7f2010c00640 10 client.15186 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.483+0000 7f2010c00640 1 -- 172.21.15.37:0/2796155341 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5567059488b0 con 0x5567059e9d80 2024-04-26T01:59:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.483+0000 7f2010c00640 10 client.15186 handle_command_reply: tid=0 2024-04-26T01:59:58.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.483+0000 7f202cfbe740 1 client.15186 shutdown 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 msgr2=0x5567059d6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059d6f80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x5567059cbec0 tx=0x7f2014011c90 comp rx=0 tx=0).stop 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 shutdown_connections 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5567059e9d80 0x5567059ec200 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059d9a00 0x5567059d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059d6f80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2796155341 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 >> 172.21.15.37:0/2796155341 conn(0x5567059cad30 msgr2=0x5567059d0a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f2010c00640 0 client.15186 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.484+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 shutdown_connections 2024-04-26T01:59:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.485+0000 7f202cfbe740 1 -- 172.21.15.37:0/2796155341 wait complete. 2024-04-26T01:59:58.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.485+0000 7f202cfbe740 20 client.15186 trim_cache size 0 max 16384 2024-04-26T01:59:58.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.513+0000 7f202cfbe740 1 Processor -- start 2024-04-26T01:59:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.513+0000 7f202cfbe740 1 -- start start 2024-04-26T01:59:58.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 0x5567059e5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059d9a00 0x5567059f2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f202cfbe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f20241997f0 con 0x5567059cf140 2024-04-26T01:59:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f202cfbe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f20240745b0 con 0x5567059d9a00 2024-04-26T01:59:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f202cfbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f20240723c0 con 0x5567059ce820 2024-04-26T01:59:58.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f2023e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 0x5567059e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f2023e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 0x5567059e5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:56320/0 (socket says 172.21.15.37:56320) 2024-04-26T01:59:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.514+0000 7f2023e00640 1 -- 172.21.15.37:0/2264372787 learned_addr learned my addr 172.21.15.37:0/2264372787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2023e00640 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059d9a00 0x5567059f2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2022a00640 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2023e00640 1 -- 172.21.15.37:0/2264372787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059d9a00 msgr2=0x5567059f2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2023e00640 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059d9a00 0x5567059f2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2023e00640 1 -- 172.21.15.37:0/2264372787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 msgr2=0x5567059f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2023e00640 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.515+0000 7f2023e00640 1 -- 172.21.15.37:0/2264372787 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567059e6880 con 0x5567059ce820 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.516+0000 7f2022a00640 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.516+0000 7f2023e00640 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 0x5567059e5310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f201c004af0 tx=0x7f201c02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1819f4a7bfb30af0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.517+0000 7f2013400640 1 -- 172.21.15.37:0/2264372787 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c030750 con 0x5567059ce820 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.517+0000 7f2013400640 1 -- 172.21.15.37:0/2264372787 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c036040 con 0x5567059ce820 2024-04-26T01:59:58.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.517+0000 7f2013400640 1 -- 172.21.15.37:0/2264372787 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c004490 con 0x5567059ce820 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 -- 172.21.15.37:0/2264372787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 msgr2=0x5567059e5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 0x5567059e5310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f201c004af0 tx=0x7f201c02a8c0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 -- 172.21.15.37:0/2264372787 shutdown_connections 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059d9a00 0x5567059f2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059cf140 0x5567059f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 --2- 172.21.15.37:0/2264372787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059ce820 0x5567059e5310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 -- 172.21.15.37:0/2264372787 >> 172.21.15.37:0/2264372787 conn(0x5567059cad30 msgr2=0x5567059d1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 -- 172.21.15.37:0/2264372787 shutdown_connections 2024-04-26T01:59:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.518+0000 7f202cfbe740 1 -- 172.21.15.37:0/2264372787 wait complete. 2024-04-26T01:59:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.519+0000 7f202cfbe740 1 -- start start 2024-04-26T01:59:58.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.519+0000 7f202cfbe740 1 Processor -- start 2024-04-26T01:59:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059e1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2024072a70 con 0x5567059d9a00 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2024073ee0 con 0x5567059ce820 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2024073140 con 0x5567059cf140 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f202cfbe740 10 client.0 _ll_register_callbacks cb 0x7f2014011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f2022a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.520+0000 7f2023e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059e1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.521+0000 7f2023e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059e1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56766/0 (socket says 172.21.15.37:56766) 2024-04-26T01:59:58.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.521+0000 7f2023e00640 1 -- 172.21.15.37:0/3821639787 learned_addr learned my addr 172.21.15.37:0/3821639787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T01:59:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.521+0000 7f2023e00640 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2023e00640 1 -- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 msgr2=0x5567059e1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2023e00640 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059e1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2023e00640 1 -- 172.21.15.37:0/3821639787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 msgr2=0x5567059dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T01:59:58.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2023e00640 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2023e00640 1 -- 172.21.15.37:0/3821639787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567059df2f0 con 0x5567059ce820 2024-04-26T01:59:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2023e00640 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059e1510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f201c009620 tx=0x7f201c004150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a06b223b319afd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T01:59:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.522+0000 7f2010c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T01:59:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f2010c00640 1 -- 172.21.15.37:0/3821639787 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c05b040 con 0x5567059ce820 2024-04-26T01:59:58.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f2010c00640 1 -- 172.21.15.37:0/3821639787 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f201c003d80 con 0x5567059ce820 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f201c036040 con 0x5567059ce820 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f2010c00640 1 -- 172.21.15.37:0/3821639787 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f201c004230 con 0x5567059ce820 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f202cfbe740 10 client.25292 fetch_fsmap learned FSMap version 19 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x556705a58650 con 0x5567059ce820 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.523+0000 7f2010c00640 1 -- 172.21.15.37:0/3821639787 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f201c036040 con 0x5567059ce820 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f2010c00640 1 -- 172.21.15.37:0/3821639787 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f201c073040 con 0x5567059ce820 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f202cfbe740 10 client.25292 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f202cfbe740 10 client.25292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T01:59:58.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f202cfbe740 20 client.25292 populate_metadata read hostname 'smithi037' 2024-04-26T01:59:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f202cfbe740 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x556705a58880 0x556705a5ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T01:59:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f202cfbe740 4 client.25292 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T01:59:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556705942320 con 0x556705a58880 2024-04-26T01:59:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.524+0000 7f2022a00640 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x556705a58880 0x556705a5ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T01:59:58.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.525+0000 7f2022a00640 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x556705a58880 0x556705a5ac70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e569131c952ad931 server_cookie=ac4855f0d0e5e893 in_seq=0 out_seq=0 2024-04-26T01:59:58.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.525+0000 7f2010c00640 10 client.25292 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:58.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.537+0000 7f2010c00640 1 -- 172.21.15.37:0/3821639787 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556705942320 con 0x556705a58880 2024-04-26T01:59:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.537+0000 7f2010c00640 10 client.25292 handle_command_reply: tid=0 2024-04-26T01:59:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.537+0000 7f202cfbe740 1 client.25292 shutdown 2024-04-26T01:59:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 msgr2=0x5567059e1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059e1510 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f201c009620 tx=0x7f201c004150 comp rx=0 tx=0).stop 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 shutdown_connections 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x556705a58880 0x556705a5ac70 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5567059d9a00 0x5567059dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5567059cf140 0x5567059e1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 --2- 172.21.15.37:0/3821639787 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5567059ce820 0x5567059e1510 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f2010c00640 0 client.25292 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T01:59:58.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.538+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 >> 172.21.15.37:0/3821639787 conn(0x5567059cad30 msgr2=0x5567059d69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.539+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 shutdown_connections 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.539+0000 7f202cfbe740 1 -- 172.21.15.37:0/3821639787 wait complete. 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.539+0000 7f202cfbe740 20 client.25292 trim_cache size 0 max 16384 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T01:59:58.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f20080a3d90 msgr2=0x7f20080a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f20080a3d90 0x7f20080a6250 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x5567059cfdc0 tx=0x7f200c00cf20 comp rx=0 tx=0).stop 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 msgr2=0x7f2024198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 0x7f2024198460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f200c00e9f0 tx=0x7f200c00eeb0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 shutdown_connections 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f20080a3d90 0x7f20080a6250 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f202419cd20 0x7f202419d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f20241072a0 0x7f20241989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 --2- 172.21.15.37:0/523985073 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f20241021d0 0x7f2024198460 secure :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f200c00e9f0 tx=0x7f200c00eeb0 comp rx=0 tx=0).stop 2024-04-26T01:59:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 >> 172.21.15.37:0/523985073 conn(0x7f20240fda10 msgr2=0x7f20240fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T01:59:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 shutdown_connections 2024-04-26T01:59:58.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T01:59:58.541+0000 7f202a200640 1 -- 172.21.15.37:0/523985073 wait complete. 2024-04-26T02:00:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:00 smithi045 ceph-mon[32520]: pgmap v64: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 83 MiB/s wr, 27 op/s 2024-04-26T02:00:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:00 smithi045 ceph-mon[32520]: overall HEALTH_OK 2024-04-26T02:00:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:00 smithi071 ceph-mon[32578]: pgmap v64: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 83 MiB/s wr, 27 op/s 2024-04-26T02:00:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:00 smithi071 ceph-mon[32578]: overall HEALTH_OK 2024-04-26T02:00:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:00 smithi037 ceph-mon[27906]: pgmap v64: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 83 MiB/s wr, 27 op/s 2024-04-26T02:00:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:00 smithi037 ceph-mon[27906]: overall HEALTH_OK 2024-04-26T02:00:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:02 smithi071 ceph-mon[32578]: pgmap v65: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 61 MiB/s wr, 18 op/s 2024-04-26T02:00:02.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:02 smithi037 ceph-mon[27906]: pgmap v65: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 61 MiB/s wr, 18 op/s 2024-04-26T02:00:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:02 smithi045 ceph-mon[32520]: pgmap v65: 97 pgs: 97 active+clean; 1.7 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 61 MiB/s wr, 18 op/s 2024-04-26T02:00:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:04 smithi071 ceph-mon[32578]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 128 MiB/s wr, 76 op/s 2024-04-26T02:00:04.443 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:04.443 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 rewriters = 275795.59 kB/sec 2024-04-26T02:00:04.443 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 rewriters = 275731.94 kB/sec 2024-04-26T02:00:04.444 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 275795.59 kB/sec 2024-04-26T02:00:04.444 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 275795.59 kB/sec 2024-04-26T02:00:04.444 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 275795.59 kB/sec 2024-04-26T02:00:04.444 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T02:00:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:04 smithi037 ceph-mon[27906]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 128 MiB/s wr, 76 op/s 2024-04-26T02:00:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:04 smithi045 ceph-mon[32520]: pgmap v66: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 128 MiB/s wr, 76 op/s 2024-04-26T02:00:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:06 smithi071 ceph-mon[32578]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-04-26T02:00:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:06 smithi037 ceph-mon[27906]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-04-26T02:00:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:06 smithi045 ceph-mon[32520]: pgmap v67: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-04-26T02:00:06.829 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:06.830 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 readers = 4985261.50 kB/sec 2024-04-26T02:00:06.830 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 readers = 4963532.85 kB/sec 2024-04-26T02:00:06.830 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 4985261.50 kB/sec 2024-04-26T02:00:06.830 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 4985261.50 kB/sec 2024-04-26T02:00:06.830 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 4985261.50 kB/sec 2024-04-26T02:00:06.830 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T02:00:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:08 smithi071 ceph-mon[32578]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-04-26T02:00:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:08 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:08.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:08 smithi037 ceph-mon[27906]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-04-26T02:00:08.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:08 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:08.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:08 smithi045 ceph-mon[32520]: pgmap v68: 97 pgs: 97 active+clean; 1.5 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 128 MiB/s wr, 75 op/s 2024-04-26T02:00:08.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:08 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 re-readers = 3291601.75 kB/sec 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 re-readers = 3282266.81 kB/sec 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 3291601.75 kB/sec 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 3291601.75 kB/sec 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 3291601.75 kB/sec 2024-04-26T02:00:09.225 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 1048576.00 kB 2024-04-26T02:00:09.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:09 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:09.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:09 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:09.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:09 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:00:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:09 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:09 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:09.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:09 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:00:09.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:09 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:00:09.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:09 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:00:09.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:09 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:00:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:10 smithi071 ceph-mon[32578]: pgmap v69: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 87 op/s 2024-04-26T02:00:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:10 smithi037 ceph-mon[27906]: pgmap v69: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 87 op/s 2024-04-26T02:00:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:10 smithi045 ceph-mon[32520]: pgmap v69: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 171 MiB/s wr, 87 op/s 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 -- 172.21.15.37:0/248232439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec102970 msgr2=0x7ffaec1069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 --2- 172.21.15.37:0/248232439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec102970 0x7ffaec1069b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009900 tx=0x7ffad802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 -- 172.21.15.37:0/248232439 shutdown_connections 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 --2- 172.21.15.37:0/248232439 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec107090 0x7ffaec10ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 --2- 172.21.15.37:0/248232439 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec102970 0x7ffaec1069b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 --2- 172.21.15.37:0/248232439 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec101fc0 0x7ffaec1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 -- 172.21.15.37:0/248232439 >> 172.21.15.37:0/248232439 conn(0x7ffaec0fd880 msgr2=0x7ffaec0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 -- 172.21.15.37:0/248232439 shutdown_connections 2024-04-26T02:00:11.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.687+0000 7ffaf3000640 1 -- 172.21.15.37:0/248232439 wait complete. 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.688+0000 7ffaf3000640 1 Processor -- start 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.688+0000 7ffaf3000640 1 -- start start 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf3000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec101fc0 0x7ffaec1a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf3000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 0x7ffaec1a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf3000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec19a670 0x7ffaec19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf3000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec111fb0 con 0x7ffaec19a670 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf3000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec111e30 con 0x7ffaec101fc0 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf3000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112130 con 0x7ffaec107090 2024-04-26T02:00:11.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 0x7ffaec1a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 0x7ffaec1a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:39042/0 (socket says 172.21.15.37:39042) 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf1200640 1 -- 172.21.15.37:0/2719064911 learned_addr learned my addr 172.21.15.37:0/2719064911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec101fc0 0x7ffaec1a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.689+0000 7ffaf2600640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec19a670 0x7ffaec19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf1200640 1 -- 172.21.15.37:0/2719064911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec101fc0 msgr2=0x7ffaec1a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf1200640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec101fc0 0x7ffaec1a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf1200640 1 -- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec19a670 msgr2=0x7ffaec19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf1200640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec19a670 0x7ffaec19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf1200640 1 -- 172.21.15.37:0/2719064911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19b270 con 0x7ffaec107090 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf1200640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 0x7ffaec1a0a50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffad8032fd0 tx=0x7ffad8031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaea000640 1 -- 172.21.15.37:0/2719064911 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad803f030 con 0x7ffaec107090 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.690+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec19b500 con 0x7ffaec107090 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.691+0000 7ffaea000640 1 -- 172.21.15.37:0/2719064911 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad8004490 con 0x7ffaec107090 2024-04-26T02:00:11.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.691+0000 7ffaea000640 1 -- 172.21.15.37:0/2719064911 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8035070 con 0x7ffaec107090 2024-04-26T02:00:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.691+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec1a75f0 con 0x7ffaec107090 2024-04-26T02:00:11.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.694+0000 7ffaf75d3740 1 Processor -- start 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.694+0000 7ffaf75d3740 1 -- start start 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaea000640 1 -- 172.21.15.37:0/2719064911 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffad8042030 con 0x7ffaec107090 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad04d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad0f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad11f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dcac6a900 con 0x562dcad052d0 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562dcac10ef0 con 0x562dcad049b0 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562dcac668c0 con 0x562dcad0fb90 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad0f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad04d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad0f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59944/0 (socket says 172.21.15.37:59944) 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf1c00640 1 -- 172.21.15.37:0/3835965673 learned_addr learned my addr 172.21.15.37:0/3835965673 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:11.704 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf1200640 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad11f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf2600640 1 -- 172.21.15.37:0/3835965673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 msgr2=0x562dcad11f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf2600640 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad11f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf2600640 1 -- 172.21.15.37:0/3835965673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 msgr2=0x562dcad0f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf2600640 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad0f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf2600640 1 -- 172.21.15.37:0/3835965673 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dca97d210 con 0x562dcad049b0 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf1200640 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.695+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad0f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.696+0000 7ffaf2600640 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad04d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009f00 tx=0x7ffae4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4389d2e50d1f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.696+0000 7ffadea00640 1 -- 172.21.15.37:0/3835965673 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4004d20 con 0x562dcad049b0 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.696+0000 7ffadea00640 1 -- 172.21.15.37:0/3835965673 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4004050 con 0x562dcad049b0 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.696+0000 7ffadea00640 1 -- 172.21.15.37:0/3835965673 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4005220 con 0x562dcad049b0 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.696+0000 7ffaea000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffad40a3d00 0x7ffad40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaea000640 1 -- 172.21.15.37:0/2719064911 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ffad80f0ac0 con 0x7ffaec107090 2024-04-26T02:00:11.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffad40a3d00 0x7ffad40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 -- 172.21.15.37:0/3835965673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 msgr2=0x562dcad04d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad04d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffae4009f00 tx=0x7ffae4011810 comp rx=0 tx=0).stop 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 -- 172.21.15.37:0/3835965673 shutdown_connections 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad11f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad0f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/3835965673 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad04d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 -- 172.21.15.37:0/3835965673 >> 172.21.15.37:0/3835965673 conn(0x562dcad00ec0 msgr2=0x562dcad012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 -- 172.21.15.37:0/3835965673 shutdown_connections 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.697+0000 7ffaf75d3740 1 -- 172.21.15.37:0/3835965673 wait complete. 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.698+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffad40a3d00 0x7ffad40a61c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x562dcad05f50 tx=0x7ffae0005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:11.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.698+0000 7ffaf75d3740 1 -- start start 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.698+0000 7ffaf75d3740 1 Processor -- start 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.698+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad0cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.698+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad052d0 0x562dcad0d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad0d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dcad127c0 con 0x562dcad0fb90 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562dcac24310 con 0x562dcad049b0 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562dcad12940 con 0x562dcad052d0 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf75d3740 10 client.0 _ll_register_callbacks cb 0x562dcabf2cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad0d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad0d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59952/0 (socket says 172.21.15.37:59952) 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf2600640 1 -- 172.21.15.37:0/545098314 learned_addr learned my addr 172.21.15.37:0/545098314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:11.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1200640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad052d0 0x562dcad0d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad0cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1200640 1 -- 172.21.15.37:0/545098314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 msgr2=0x562dcad0cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1200640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad0cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1200640 1 -- 172.21.15.37:0/545098314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 msgr2=0x562dcad0d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1200640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad0d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf1200640 1 -- 172.21.15.37:0/545098314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcaa49620 con 0x562dcad052d0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.699+0000 7ffaf2600640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad0d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffaf1200640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad052d0 0x562dcad0d020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffad80ee240 tx=0x7ffad8009630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5832067f17f4d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffad3400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffad3400640 1 -- 172.21.15.37:0/545098314 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8049050 con 0x562dcad052d0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae4004050 con 0x562dcad052d0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffad3400640 1 -- 172.21.15.37:0/545098314 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ffad80ee4c0 con 0x562dcad052d0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffad3400640 1 -- 172.21.15.37:0/545098314 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ffad80bc4a0 con 0x562dcad052d0 2024-04-26T02:00:11.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffaf75d3740 10 client.25345 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562dcad127c0 con 0x562dcad052d0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.700+0000 7ffad3400640 1 -- 172.21.15.37:0/545098314 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffae4004050 con 0x562dcad052d0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffad3400640 1 -- 172.21.15.37:0/545098314 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ffad80c5270 con 0x562dcad052d0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf75d3740 10 client.25345 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf75d3740 10 client.25345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf75d3740 20 client.25345 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad1fda0 0x562dcad221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf75d3740 4 client.25345 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dcac7e8b0 con 0x562dcad1fda0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.701+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad1fda0 0x562dcad221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.702+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad1fda0 0x562dcad221f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e2e8b78eade931 server_cookie=7761f132a0be52cc in_seq=0 out_seq=0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.702+0000 7ffad3400640 10 client.25345 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffad3400640 1 -- 172.21.15.37:0/545098314 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dcac7e8b0 con 0x562dcad1fda0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffad3400640 10 client.25345 handle_command_reply: tid=0 2024-04-26T02:00:11.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 client.25345 shutdown 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad052d0 msgr2=0x562dcad0d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad052d0 0x562dcad0d020 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffad80ee240 tx=0x7ffad8009630 comp rx=0 tx=0).stop 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 shutdown_connections 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad1fda0 0x562dcad221f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad0d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad052d0 0x562dcad0d020 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffad3400640 0 client.25345 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/545098314 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad0cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 >> 172.21.15.37:0/545098314 conn(0x562dcad00ec0 msgr2=0x562dcad06c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 shutdown_connections 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 1 -- 172.21.15.37:0/545098314 wait complete. 2024-04-26T02:00:11.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.703+0000 7ffaf75d3740 20 client.25345 trim_cache size 0 max 16384 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 Processor -- start 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 -- start start 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad049b0 0x562dcad1b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad052d0 0x562dcad0d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad27b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1adaf0 con 0x562dcad0fb90 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1b0370 con 0x562dcad052d0 2024-04-26T02:00:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.720+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ae160 con 0x562dcad049b0 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad27b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad049b0 0x562dcad1b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad052d0 0x562dcad0d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf2600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad27b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59966/0 (socket says 172.21.15.37:59966) 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf2600640 1 -- 172.21.15.37:0/1431789083 learned_addr learned my addr 172.21.15.37:0/1431789083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1200640 1 -- 172.21.15.37:0/1431789083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad049b0 msgr2=0x562dcad1b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1200640 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad049b0 0x562dcad1b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1200640 1 -- 172.21.15.37:0/1431789083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 msgr2=0x562dcad27b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1200640 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad27b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1200640 1 -- 172.21.15.37:0/1431789083 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dcad22870 con 0x562dcad052d0 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.721+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad049b0 0x562dcad1b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf1200640 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad052d0 0x562dcad0d1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffad80b9510 tx=0x7ffad803b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8d66e2d885fdc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffade000640 1 -- 172.21.15.37:0/1431789083 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80d5070 con 0x562dcad052d0 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffade000640 1 -- 172.21.15.37:0/1431789083 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad80bcb40 con 0x562dcad052d0 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffade000640 1 -- 172.21.15.37:0/1431789083 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8049050 con 0x562dcad052d0 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1431789083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad052d0 msgr2=0x562dcad0d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad052d0 0x562dcad0d1b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffad80b9510 tx=0x7ffad803b560 comp rx=0 tx=0).stop 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1431789083 shutdown_connections 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad0fb90 0x562dcad27b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad052d0 0x562dcad0d1b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1431789083 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad049b0 0x562dcad1b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.722+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1431789083 >> 172.21.15.37:0/1431789083 conn(0x562dcad00ec0 msgr2=0x562dcad0caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.723+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1431789083 shutdown_connections 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.723+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1431789083 wait complete. 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.723+0000 7ffaf75d3740 1 -- start start 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.723+0000 7ffaf75d3740 1 Processor -- start 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad18030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad153c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ae830 con 0x562dcad052d0 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1afca0 con 0x562dcad049b0 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1aef00 con 0x562dcad0fb90 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf75d3740 10 client.0 _ll_register_callbacks cb 0x7ffad8009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:11.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad153c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad153c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59978/0 (socket says 172.21.15.37:59978) 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 -- 172.21.15.37:0/1856146416 learned_addr learned my addr 172.21.15.37:0/1856146416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad18030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf2600640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 -- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 msgr2=0x562dcad159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad159d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 -- 172.21.15.37:0/1856146416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 msgr2=0x562dcad18030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad18030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.724+0000 7ffaf1200640 1 -- 172.21.15.37:0/1856146416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcad11f80 con 0x562dcad052d0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffaf1200640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad153c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffad8031440 tx=0x7ffad80bb050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57d306465f5dfd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffad3400640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffad3400640 1 -- 172.21.15.37:0/1856146416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad80d5070 con 0x562dcad052d0 2024-04-26T02:00:11.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffad3400640 1 -- 172.21.15.37:0/1856146416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ffad80b8890 con 0x562dcad052d0 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffad80bcb40 con 0x562dcad052d0 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffad3400640 1 -- 172.21.15.37:0/1856146416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ffad8049070 con 0x562dcad052d0 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffaf75d3740 10 client.15210 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562dcad11f80 con 0x562dcad052d0 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.725+0000 7ffad3400640 1 -- 172.21.15.37:0/1856146416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffad80bcb40 con 0x562dcad052d0 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffad3400640 1 -- 172.21.15.37:0/1856146416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ffad80c1fd0 con 0x562dcad052d0 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf75d3740 10 client.15210 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf75d3740 10 client.15210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf75d3740 20 client.15210 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad865c0 0x562dcad889e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf75d3740 4 client.15210 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:11.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562dcac78320 con 0x562dcad865c0 2024-04-26T02:00:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.726+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad865c0 0x562dcad889e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.727+0000 7ffaf1c00640 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad865c0 0x562dcad889e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f85f918cb31f0aaa server_cookie=672cd60f44813f69 in_seq=0 out_seq=0 2024-04-26T02:00:11.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.727+0000 7ffad3400640 10 client.15210 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffad3400640 1 -- 172.21.15.37:0/1856146416 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562dcac78320 con 0x562dcad865c0 2024-04-26T02:00:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffad3400640 10 client.15210 handle_command_reply: tid=0 2024-04-26T02:00:11.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 client.15210 shutdown 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 msgr2=0x562dcad153c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad153c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ffad8031440 tx=0x7ffad80bb050 comp rx=0 tx=0).stop 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 shutdown_connections 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562dcad865c0 0x562dcad889e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562dcad0fb90 0x562dcad159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dcad052d0 0x562dcad153c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 --2- 172.21.15.37:0/1856146416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562dcad049b0 0x562dcad18030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 >> 172.21.15.37:0/1856146416 conn(0x562dcad00ec0 msgr2=0x562dcad06360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffad3400640 0 client.15210 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 shutdown_connections 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 1 -- 172.21.15.37:0/1856146416 wait complete. 2024-04-26T02:00:11.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.738+0000 7ffaf75d3740 20 client.15210 trim_cache size 0 max 16384 2024-04-26T02:00:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:00:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:00:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:00:11.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffad40a3d00 msgr2=0x7ffad40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffad40a3d00 0x7ffad40a61c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x562dcad05f50 tx=0x7ffae0005f90 comp rx=0 tx=0).stop 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 msgr2=0x7ffaec1a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 0x7ffaec1a0a50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ffad8032fd0 tx=0x7ffad8031a20 comp rx=0 tx=0).stop 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 shutdown_connections 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffad40a3d00 0x7ffad40a61c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaec19a670 0x7ffaec19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffaec107090 0x7ffaec1a0a50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 --2- 172.21.15.37:0/2719064911 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffaec101fc0 0x7ffaec1a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 >> 172.21.15.37:0/2719064911 conn(0x7ffaec0fd880 msgr2=0x7ffaec0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 shutdown_connections 2024-04-26T02:00:11.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:11.739+0000 7ffaf3000640 1 -- 172.21.15.37:0/2719064911 wait complete. 2024-04-26T02:00:12.306 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:12 smithi045 ceph-mon[32520]: pgmap v70: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 110 MiB/s wr, 69 op/s 2024-04-26T02:00:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:12 smithi071 ceph-mon[32578]: pgmap v70: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 110 MiB/s wr, 69 op/s 2024-04-26T02:00:12.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:12 smithi037 ceph-mon[27906]: pgmap v70: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 110 MiB/s wr, 69 op/s 2024-04-26T02:00:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:14 smithi071 ceph-mon[32578]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 110 MiB/s wr, 73 op/s 2024-04-26T02:00:14.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:14 smithi037 ceph-mon[27906]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 110 MiB/s wr, 73 op/s 2024-04-26T02:00:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:14 smithi045 ceph-mon[32520]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 110 MiB/s wr, 73 op/s 2024-04-26T02:00:15.729 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:15.730 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:15.730 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:15.730 INFO:tasks.workunit.client.0.smithi037.stdout:iozone test complete. 2024-04-26T02:00:15.732 INFO:tasks.workunit.client.0.smithi037.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-26T02:00:15.824 INFO:tasks.workunit.client.0.smithi037.stdout: Iozone: Performance Test of File I/O 2024-04-26T02:00:15.824 INFO:tasks.workunit.client.0.smithi037.stdout: Version $Revision: 3.494 $ 2024-04-26T02:00:15.824 INFO:tasks.workunit.client.0.smithi037.stdout: Compiled for 64 bit mode. 2024-04-26T02:00:15.824 INFO:tasks.workunit.client.0.smithi037.stdout: Build: linux-AMD64 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Alexey Skidanov, Sudhir Kumar. 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: Run began: Fri Apr 26 02:00:15 2024 2024-04-26T02:00:15.825 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Include close in write timing 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Include fsync in write timing 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: File size set to 10485760 kB 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Record Size 1024 kB 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Output is in kBytes/sec 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Time Resolution = 0.000001 seconds. 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Processor cache size set to 1024 kBytes. 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Processor cache line size set to 32 bytes. 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: File stride size set to 17 * record size. 2024-04-26T02:00:15.826 INFO:tasks.workunit.client.0.smithi037.stdout: Throughput test with 1 process 2024-04-26T02:00:15.827 INFO:tasks.workunit.client.0.smithi037.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-04-26T02:00:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:16 smithi071 ceph-mon[32578]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 43 MiB/s wr, 15 op/s 2024-04-26T02:00:16.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:16 smithi037 ceph-mon[27906]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 43 MiB/s wr, 15 op/s 2024-04-26T02:00:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:16 smithi045 ceph-mon[32520]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 43 MiB/s wr, 15 op/s 2024-04-26T02:00:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:18 smithi071 ceph-mon[32578]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 43 MiB/s wr, 15 op/s 2024-04-26T02:00:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:18 smithi037 ceph-mon[27906]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 43 MiB/s wr, 15 op/s 2024-04-26T02:00:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:18 smithi045 ceph-mon[32520]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 43 MiB/s wr, 15 op/s 2024-04-26T02:00:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:20 smithi071 ceph-mon[32578]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 43 MiB/s wr, 18 op/s 2024-04-26T02:00:20.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:20 smithi037 ceph-mon[27906]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 43 MiB/s wr, 18 op/s 2024-04-26T02:00:20.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:20 smithi045 ceph-mon[32520]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 43 MiB/s wr, 18 op/s 2024-04-26T02:00:20.674 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T02:00:20.675 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T02:00:20.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:00:21.473 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:22 smithi037 ceph-mon[27906]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-04-26T02:00:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:22 smithi071 ceph-mon[32578]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-04-26T02:00:22.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:22 smithi045 ceph-mon[32520]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 12 KiB/s wr, 6 op/s 2024-04-26T02:00:24.271 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:24 smithi037 ceph-mon[27906]: pgmap v76: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 102 MiB/s wr, 56 op/s 2024-04-26T02:00:24.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 -- 172.21.15.37:0/1765038102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406b0f0 msgr2=0x7f129410f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 --2- 172.21.15.37:0/1765038102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406b0f0 0x7f129410f900 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f128c009f90 tx=0x7f128c00b330 comp rx=0 tx=0).stop 2024-04-26T02:00:24.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 -- 172.21.15.37:0/1765038102 shutdown_connections 2024-04-26T02:00:24.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 --2- 172.21.15.37:0/1765038102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1294108a30 0x7f1294108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 --2- 172.21.15.37:0/1765038102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406b0f0 0x7f129410f900 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 --2- 172.21.15.37:0/1765038102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f129406a750 0x7f129406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 -- 172.21.15.37:0/1765038102 >> 172.21.15.37:0/1765038102 conn(0x7f12940788f0 msgr2=0x7f1294078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 -- 172.21.15.37:0/1765038102 shutdown_connections 2024-04-26T02:00:24.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.384+0000 7f129c400700 1 -- 172.21.15.37:0/1765038102 wait complete. 2024-04-26T02:00:24.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.385+0000 7f129c400700 1 Processor -- start 2024-04-26T02:00:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.385+0000 7f129c400700 1 -- start start 2024-04-26T02:00:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f129c400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 0x7f1294101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f129c400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129406b0f0 0x7f1294101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f129c400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294108a30 0x7f1294102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f129c400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1294112c20 con 0x7f129406b0f0 2024-04-26T02:00:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f129c400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f1294112aa0 con 0x7f1294108a30 2024-04-26T02:00:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f129c400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f1294112da0 con 0x7f129406a750 2024-04-26T02:00:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f1299a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 0x7f1294101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f1299000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129406b0f0 0x7f1294101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f1299a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 0x7f1294101a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50020/0 (socket says 172.21.15.37:50020) 2024-04-26T02:00:24.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.386+0000 7f1299a00700 1 -- 172.21.15.37:0/460824196 learned_addr learned my addr 172.21.15.37:0/460824196 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:24.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f129a400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294108a30 0x7f1294102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f1299a00700 1 -- 172.21.15.37:0/460824196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294108a30 msgr2=0x7f1294102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f1299a00700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294108a30 0x7f1294102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f1299a00700 1 -- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129406b0f0 msgr2=0x7f1294101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f1299a00700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129406b0f0 0x7f1294101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f1299a00700 1 -- 172.21.15.37:0/460824196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12941a7610 con 0x7f129406a750 2024-04-26T02:00:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.387+0000 7f1299a00700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 0x7f1294101a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f128800e9a0 tx=0x7f128800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.388+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128800c9c0 con 0x7f129406a750 2024-04-26T02:00:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.388+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128800cb60 con 0x7f129406a750 2024-04-26T02:00:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.388+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288018680 con 0x7f129406a750 2024-04-26T02:00:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.388+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12941a7900 con 0x7f129406a750 2024-04-26T02:00:24.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.388+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12941a7df0 con 0x7f129406a750 2024-04-26T02:00:24.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.389+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1294069730 con 0x7f129406a750 2024-04-26T02:00:24.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.391+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1288010430 con 0x7f129406a750 2024-04-26T02:00:24.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.393+0000 7f1292000700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f127c0a3e30 0x7f127c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.393+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f12880d0d10 con 0x7f129406a750 2024-04-26T02:00:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.393+0000 7f1299000700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f127c0a3e30 0x7f127c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.394+0000 7f1299000700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f127c0a3e30 0x7f127c0a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1294108030 tx=0x7f1280009310 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.395+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f128809d380 con 0x7f129406a750 2024-04-26T02:00:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:24 smithi071 ceph-mon[32578]: pgmap v76: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 102 MiB/s wr, 56 op/s 2024-04-26T02:00:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:24 smithi045 ceph-mon[32520]: pgmap v76: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 102 MiB/s wr, 56 op/s 2024-04-26T02:00:24.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.860+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1294102ef0 con 0x7f129406a750 2024-04-26T02:00:24.861 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:00:24.861 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:00:24.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.861+0000 7f1292000700 1 -- 172.21.15.37:0/460824196 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f128809cc80 con 0x7f129406a750 2024-04-26T02:00:24.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.866+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f127c0a3e30 msgr2=0x7f127c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.866+0000 7f129c400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f127c0a3e30 0x7f127c0a62f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1294108030 tx=0x7f1280009310 comp rx=0 tx=0).stop 2024-04-26T02:00:24.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.866+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 msgr2=0x7f1294101a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.866+0000 7f129c400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 0x7f1294101a10 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f128800e9a0 tx=0x7f128800ecb0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 shutdown_connections 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f127c0a3e30 0x7f127c0a62f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f1294108a30 0x7f1294102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f129406a750 0x7f1294101a10 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 --2- 172.21.15.37:0/460824196 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f129406b0f0 0x7f1294101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 >> 172.21.15.37:0/460824196 conn(0x7f12940788f0 msgr2=0x7f12940ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.867+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 shutdown_connections 2024-04-26T02:00:24.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:24.868+0000 7f129c400700 1 -- 172.21.15.37:0/460824196 wait complete. 2024-04-26T02:00:24.868 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 -- 172.21.15.37:0/1892511537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 msgr2=0x7fe504115040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 --2- 172.21.15.37:0/1892511537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe504115040 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009ca0 tx=0x7fe4fc02a8c0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 -- 172.21.15.37:0/1892511537 shutdown_connections 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 --2- 172.21.15.37:0/1892511537 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe504115040 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 --2- 172.21.15.37:0/1892511537 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504100920 0x7fe504100da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 --2- 172.21.15.37:0/1892511537 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504107f00 0x7fe5041003e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 -- 172.21.15.37:0/1892511537 >> 172.21.15.37:0/1892511537 conn(0x7fe504075d60 msgr2=0x7fe504076170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 -- 172.21.15.37:0/1892511537 shutdown_connections 2024-04-26T02:00:24.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.916+0000 7fe50aa00640 1 -- 172.21.15.37:0/1892511537 wait complete. 2024-04-26T02:00:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.917+0000 7fe50aa00640 1 Processor -- start 2024-04-26T02:00:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.917+0000 7fe50aa00640 1 -- start start 2024-04-26T02:00:24.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe50aa00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504100920 0x7fe5041a46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe50aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 0x7fe5041a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe50aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe50419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe50aa00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe504118800 con 0x7fe504100920 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe50aa00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe504118680 con 0x7fe504112c50 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe50aa00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe504118980 con 0x7fe504107f00 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe503400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 0x7fe5041a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe503400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 0x7fe5041a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50030/0 (socket says 172.21.15.37:50030) 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe503400640 1 -- 172.21.15.37:0/346801221 learned_addr learned my addr 172.21.15.37:0/346801221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.918+0000 7fe503e00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504100920 0x7fe5041a46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe508a00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe50419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503400640 1 -- 172.21.15.37:0/346801221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 msgr2=0x7fe50419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503400640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe50419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503400640 1 -- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504100920 msgr2=0x7fe5041a46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503400640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504100920 0x7fe5041a46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503400640 1 -- 172.21.15.37:0/346801221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50419f020 con 0x7fe504107f00 2024-04-26T02:00:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe508a00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe50419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503e00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504100920 0x7fe5041a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:24.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.919+0000 7fe503400640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 0x7fe5041a4c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e9f0 tx=0x7fe4f000edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.920+0000 7fe500c00640 1 -- 172.21.15.37:0/346801221 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f000ccc0 con 0x7fe504107f00 2024-04-26T02:00:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.920+0000 7fe500c00640 1 -- 172.21.15.37:0/346801221 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f000ce60 con 0x7fe504107f00 2024-04-26T02:00:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.920+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50419f310 con 0x7fe504107f00 2024-04-26T02:00:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.920+0000 7fe500c00640 1 -- 172.21.15.37:0/346801221 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0010430 con 0x7fe504107f00 2024-04-26T02:00:24.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.920+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5041ab5b0 con 0x7fe504107f00 2024-04-26T02:00:24.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.924+0000 7fe50d839740 1 Processor -- start 2024-04-26T02:00:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.924+0000 7fe50d839740 1 -- start start 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.924+0000 7fe50d839740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876529820 0x560876529c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe50d839740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x5608765344c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe50d839740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x560876536df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe50d839740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56087648f900 con 0x56087652a140 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe50d839740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560876435ef0 con 0x560876534a00 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe50d839740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x56087648b8c0 con 0x560876529820 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe503e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x5608765344c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe508a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876529820 0x560876529c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe508a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876529820 0x560876529c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:50044/0 (socket says 172.21.15.37:50044) 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe508a00640 1 -- 172.21.15.37:0/4193650499 learned_addr learned my addr 172.21.15.37:0/4193650499 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe503e00640 1 -- 172.21.15.37:0/4193650499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876529820 msgr2=0x560876529c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe503e00640 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876529820 0x560876529c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe503e00640 1 -- 172.21.15.37:0/4193650499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 msgr2=0x560876536df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:00:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe503e00640 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x560876536df0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.925+0000 7fe503e00640 1 -- 172.21.15.37:0/4193650499 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608761a2210 con 0x56087652a140 2024-04-26T02:00:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.926+0000 7fe500c00640 1 -- 172.21.15.37:0/346801221 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe4f0010610 con 0x7fe504107f00 2024-04-26T02:00:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.926+0000 7fe503e00640 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x5608765344c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400af10 tx=0x7fe4f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f74d979031009114 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.926+0000 7fe4f9200640 1 -- 172.21.15.37:0/4193650499 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4005220 con 0x56087652a140 2024-04-26T02:00:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.926+0000 7fe4f9200640 1 -- 172.21.15.37:0/4193650499 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4004c50 con 0x56087652a140 2024-04-26T02:00:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.927+0000 7fe500c00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fe4e80a3d10 0x7fe4e80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.927+0000 7fe50d839740 1 -- 172.21.15.37:0/4193650499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 msgr2=0x5608765344c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.927+0000 7fe50d839740 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x5608765344c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400af10 tx=0x7fe4f400f790 comp rx=0 tx=0).stop 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.927+0000 7fe500c00640 1 -- 172.21.15.37:0/346801221 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe4f00d0cf0 con 0x7fe504107f00 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 -- 172.21.15.37:0/4193650499 shutdown_connections 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x560876536df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x5608765344c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 --2- 172.21.15.37:0/4193650499 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876529820 0x560876529c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 -- 172.21.15.37:0/4193650499 >> 172.21.15.37:0/4193650499 conn(0x560876525d30 msgr2=0x560876526110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe508a00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fe4e80a3d10 0x7fe4e80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 -- 172.21.15.37:0/4193650499 shutdown_connections 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.928+0000 7fe50d839740 1 -- 172.21.15.37:0/4193650499 wait complete. 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.929+0000 7fe50d839740 1 -- start start 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.929+0000 7fe508a00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fe4e80a3d10 0x7fe4e80a61d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009430 tx=0x7fe4fc009580 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.929+0000 7fe50d839740 1 Processor -- start 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876529820 0x5608765318a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x560876531de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x560876532320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560876537630 con 0x56087652a140 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x560876449310 con 0x560876529820 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5608765377b0 con 0x560876534a00 2024-04-26T02:00:24.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe50d839740 10 client.0 _ll_register_callbacks cb 0x560876417cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe503400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x560876531de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe503400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x560876531de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51798/0 (socket says 172.21.15.37:51798) 2024-04-26T02:00:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe503400640 1 -- 172.21.15.37:0/2964755180 learned_addr learned my addr 172.21.15.37:0/2964755180 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.930+0000 7fe508a00640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x560876532320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.930 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503e00640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876529820 0x5608765318a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503400640 1 -- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 msgr2=0x560876532320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503400640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x560876532320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503400640 1 -- 172.21.15.37:0/2964755180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876529820 msgr2=0x5608765318a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503400640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876529820 0x5608765318a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503400640 1 -- 172.21.15.37:0/2964755180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56087626e620 con 0x56087652a140 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe503400640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x560876531de0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c6a0 tx=0x7fe4f009d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf5157b08d6e861 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.931+0000 7fe4ed600640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe4ed600640 1 -- 172.21.15.37:0/2964755180 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00046c0 con 0x56087652a140 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe4ed600640 1 -- 172.21.15.37:0/2964755180 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe4f00bd070 con 0x56087652a140 2024-04-26T02:00:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4004c50 con 0x56087652a140 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe4ed600640 1 -- 172.21.15.37:0/2964755180 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe4f0004860 con 0x56087652a140 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe50d839740 10 client.15225 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5608765438f0 con 0x56087652a140 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe4ed600640 1 -- 172.21.15.37:0/2964755180 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f4004c50 con 0x56087652a140 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.932+0000 7fe4ed600640 1 -- 172.21.15.37:0/2964755180 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fe4f00ceb60 con 0x56087652a140 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe50d839740 10 client.15225 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe50d839740 10 client.15225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe50d839740 20 client.15225 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe50d839740 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x560876544d60 0x560876547150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe50d839740 4 client.15225 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608764a38b0 con 0x560876544d60 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.933+0000 7fe503e00640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x560876544d60 0x560876547150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.934+0000 7fe503e00640 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x560876544d60 0x560876547150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f35e60a7a4c172b0 server_cookie=b94d64caaaf5cef2 in_seq=0 out_seq=0 2024-04-26T02:00:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.934+0000 7fe4ed600640 10 client.15225 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.934+0000 7fe4ed600640 1 -- 172.21.15.37:0/2964755180 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608764a38b0 con 0x560876544d60 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.934+0000 7fe4ed600640 10 client.15225 handle_command_reply: tid=0 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.934+0000 7fe50d839740 1 client.15225 shutdown 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 msgr2=0x560876531de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x560876531de0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000c6a0 tx=0x7fe4f009d4a0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 shutdown_connections 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x560876544d60 0x560876547150 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x560876532320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56087652a140 0x560876531de0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 --2- 172.21.15.37:0/2964755180 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876529820 0x5608765318a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 >> 172.21.15.37:0/2964755180 conn(0x560876525d30 msgr2=0x56087652ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe4ed600640 0 client.15225 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 shutdown_connections 2024-04-26T02:00:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 1 -- 172.21.15.37:0/2964755180 wait complete. 2024-04-26T02:00:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.935+0000 7fe50d839740 20 client.15225 trim_cache size 0 max 16384 2024-04-26T02:00:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.953+0000 7fe50d839740 1 Processor -- start 2024-04-26T02:00:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.953+0000 7fe50d839740 1 -- start start 2024-04-26T02:00:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe50d839740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876525610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe50d839740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 0x560876532140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe50d839740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x56087654ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe50d839740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b1e60 con 0x560876529820 2024-04-26T02:00:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe50d839740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b46e0 con 0x56087652a140 2024-04-26T02:00:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe50d839740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b24d0 con 0x560876534a00 2024-04-26T02:00:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876525610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 0x560876532140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 0x560876532140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:39712/0 (socket says 172.21.15.37:39712) 2024-04-26T02:00:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 -- 172.21.15.37:0/3225698879 learned_addr learned my addr 172.21.15.37:0/3225698879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:24.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe508a00640 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x56087654ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 -- 172.21.15.37:0/3225698879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 msgr2=0x56087654ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x56087654ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 -- 172.21.15.37:0/3225698879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 msgr2=0x560876525610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.954+0000 7fe503400640 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876525610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.955+0000 7fe503400640 1 -- 172.21.15.37:0/3225698879 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560876525b50 con 0x56087652a140 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.955+0000 7fe503400640 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 0x560876532140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e9c0 tx=0x7fe4f0005730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7205413da5e9117a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.955+0000 7fe4efe00640 1 -- 172.21.15.37:0/3225698879 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f0068070 con 0x56087652a140 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.955+0000 7fe4efe00640 1 -- 172.21.15.37:0/3225698879 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f00a5440 con 0x56087652a140 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.955+0000 7fe4efe00640 1 -- 172.21.15.37:0/3225698879 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f00ceb60 con 0x56087652a140 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 -- 172.21.15.37:0/3225698879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 msgr2=0x560876532140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 0x560876532140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e9c0 tx=0x7fe4f0005730 comp rx=0 tx=0).stop 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 -- 172.21.15.37:0/3225698879 shutdown_connections 2024-04-26T02:00:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x560876534a00 0x56087654ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x56087652a140 0x560876532140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 --2- 172.21.15.37:0/3225698879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876525610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 -- 172.21.15.37:0/3225698879 >> 172.21.15.37:0/3225698879 conn(0x560876525d30 msgr2=0x5608765301b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 -- 172.21.15.37:0/3225698879 shutdown_connections 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 -- 172.21.15.37:0/3225698879 wait complete. 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.956+0000 7fe50d839740 1 -- start start 2024-04-26T02:00:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 Processor -- start 2024-04-26T02:00:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876539a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56087652a140 0x560876539f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x5608765a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b2ba0 con 0x560876529820 2024-04-26T02:00:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b4010 con 0x560876534a00 2024-04-26T02:00:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fe5041b3270 con 0x56087652a140 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe50d839740 10 client.0 _ll_register_callbacks cb 0x7fe4f009c5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe503e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876539a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe508a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x5608765a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe503400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56087652a140 0x560876539f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe508a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x5608765a92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:39720/0 (socket says 172.21.15.37:39720) 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.957+0000 7fe503e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876539a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51826/0 (socket says 172.21.15.37:51826) 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 -- 172.21.15.37:0/938176915 learned_addr learned my addr 172.21.15.37:0/938176915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 -- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56087652a140 msgr2=0x560876539f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56087652a140 0x560876539f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 -- 172.21.15.37:0/938176915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 msgr2=0x560876539a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876539a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 -- 172.21.15.37:0/938176915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608765a97f0 con 0x560876534a00 2024-04-26T02:00:24.960 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe503e00640 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876539a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe508a00640 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x5608765a92b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009bf0 tx=0x7fe4fc02a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d43fc779fedfad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.958+0000 7fe4ed600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe4ed600640 1 -- 172.21.15.37:0/938176915 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc004250 con 0x560876534a00 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f00a5440 con 0x560876534a00 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe4ed600640 1 -- 172.21.15.37:0/938176915 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fe4fc041030 con 0x560876534a00 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe4ed600640 1 -- 172.21.15.37:0/938176915 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fe4fc0043f0 con 0x560876534a00 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 10 client.25328 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5608765aa8b0 con 0x560876534a00 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe4ed600640 1 -- 172.21.15.37:0/938176915 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4f00a5440 con 0x560876534a00 2024-04-26T02:00:24.961 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe4ed600640 1 -- 172.21.15.37:0/938176915 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fe4fc0305c0 con 0x560876534a00 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 10 client.25328 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 10 client.25328 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 20 client.25328 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5608765ab010 0x5608765ad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 4 client.25328 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.959+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56087649d320 con 0x5608765ab010 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.960+0000 7fe503e00640 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5608765ab010 0x5608765ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.960+0000 7fe503e00640 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5608765ab010 0x5608765ad430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2a3360de7da3299 server_cookie=e5279f2d83ee996 in_seq=0 out_seq=0 2024-04-26T02:00:24.962 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.960+0000 7fe4ed600640 10 client.25328 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe4ed600640 1 -- 172.21.15.37:0/938176915 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56087649d320 con 0x5608765ab010 2024-04-26T02:00:24.981 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe4ed600640 10 client.25328 handle_command_reply: tid=0 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 client.25328 shutdown 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 msgr2=0x5608765a92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x5608765a92b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009bf0 tx=0x7fe4fc02a8c0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 shutdown_connections 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5608765ab010 0x5608765ad430 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x560876534a00 0x5608765a92b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x56087652a140 0x560876539f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 --2- 172.21.15.37:0/938176915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560876529820 0x560876539a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 >> 172.21.15.37:0/938176915 conn(0x560876525d30 msgr2=0x56087652b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.982 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.983+0000 7fe4ed600640 0 client.25328 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.984+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 shutdown_connections 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.984+0000 7fe50d839740 1 -- 172.21.15.37:0/938176915 wait complete. 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.984+0000 7fe50d839740 20 client.25328 trim_cache size 0 max 16384 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:00:24.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:00:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fe4e80a3d10 msgr2=0x7fe4e80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fe4e80a3d10 0x7fe4e80a61d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009430 tx=0x7fe4fc009580 comp rx=0 tx=0).stop 2024-04-26T02:00:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 msgr2=0x7fe5041a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:24.987 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 0x7fe5041a4c30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe4f000e9f0 tx=0x7fe4f000edf0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 shutdown_connections 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fe4e80a3d10 0x7fe4e80a61d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fe504112c50 0x7fe50419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fe504107f00 0x7fe5041a4c30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 --2- 172.21.15.37:0/346801221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe504100920 0x7fe5041a46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 >> 172.21.15.37:0/346801221 conn(0x7fe504075d60 msgr2=0x7fe504111880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.989+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 shutdown_connections 2024-04-26T02:00:24.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:24.990+0000 7fe50aa00640 1 -- 172.21.15.37:0/346801221 wait complete. 2024-04-26T02:00:25.937 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T02:00:25.937 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T02:00:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:25 smithi045 ceph-mon[32520]: pgmap v77: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 102 MiB/s wr, 52 op/s 2024-04-26T02:00:26.004 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:25 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/460824196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:26.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:25 smithi037 ceph-mon[27906]: pgmap v77: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 102 MiB/s wr, 52 op/s 2024-04-26T02:00:26.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:25 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/460824196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:25 smithi071 ceph-mon[32578]: pgmap v77: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 102 MiB/s wr, 52 op/s 2024-04-26T02:00:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:25 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/460824196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:00:26.637 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:00:28.039 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:27 smithi037 ceph-mon[27906]: pgmap v78: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 102 MiB/s wr, 52 op/s 2024-04-26T02:00:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:27 smithi071 ceph-mon[32578]: pgmap v78: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 102 MiB/s wr, 52 op/s 2024-04-26T02:00:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:27 smithi045 ceph-mon[32520]: pgmap v78: 97 pgs: 97 active+clean; 2.2 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 102 MiB/s wr, 52 op/s 2024-04-26T02:00:30.075 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:29 smithi037 ceph-mon[27906]: pgmap v79: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 242 MiB/s wr, 93 op/s 2024-04-26T02:00:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:29 smithi071 ceph-mon[32578]: pgmap v79: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 242 MiB/s wr, 93 op/s 2024-04-26T02:00:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:29 smithi045 ceph-mon[32520]: pgmap v79: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 242 MiB/s wr, 93 op/s 2024-04-26T02:00:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 -- 172.21.15.37:0/3314147905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 msgr2=0x7fcb30102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 --2- 172.21.15.37:0/3314147905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 0x7fcb30102850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb20009990 tx=0x7fcb20009ca0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 -- 172.21.15.37:0/3314147905 shutdown_connections 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 --2- 172.21.15.37:0/3314147905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb3010f4f0 0x7fcb301118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 --2- 172.21.15.37:0/3314147905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 0x7fcb30102850 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 --2- 172.21.15.37:0/3314147905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb30102d90 0x7fcb3010efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 -- 172.21.15.37:0/3314147905 >> 172.21.15.37:0/3314147905 conn(0x7fcb300788e0 msgr2=0x7fcb30078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.314+0000 7fcb35600700 1 -- 172.21.15.37:0/3314147905 shutdown_connections 2024-04-26T02:00:30.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.315+0000 7fcb35600700 1 -- 172.21.15.37:0/3314147905 wait complete. 2024-04-26T02:00:30.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.316+0000 7fcb35600700 1 Processor -- start 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.316+0000 7fcb35600700 1 -- start start 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.317+0000 7fcb35600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 0x7fcb30103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.317+0000 7fcb35600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb30102d90 0x7fcb30104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.317+0000 7fcb35600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3010f4f0 0x7fcb301046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.317+0000 7fcb35600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30114e90 con 0x7fcb30102d90 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.318+0000 7fcb35600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30114d10 con 0x7fcb3010f4f0 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.318+0000 7fcb35600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb30115010 con 0x7fcb30102470 2024-04-26T02:00:30.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.319+0000 7fcb2e000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb30102d90 0x7fcb30104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.319+0000 7fcb2f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3010f4f0 0x7fcb301046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.319+0000 7fcb2f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3010f4f0 0x7fcb301046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:42436/0 (socket says 172.21.15.37:42436) 2024-04-26T02:00:30.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.319+0000 7fcb2f400700 1 -- 172.21.15.37:0/3252743659 learned_addr learned my addr 172.21.15.37:0/3252743659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.321+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 0x7fcb30103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.321+0000 7fcb2e000700 1 -- 172.21.15.37:0/3252743659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 msgr2=0x7fcb30103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.321+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 0x7fcb30103c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.321+0000 7fcb2e000700 1 -- 172.21.15.37:0/3252743659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3010f4f0 msgr2=0x7fcb301046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.321+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3010f4f0 0x7fcb301046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.321+0000 7fcb2e000700 1 -- 172.21.15.37:0/3252743659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb3019b140 con 0x7fcb30102d90 2024-04-26T02:00:30.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.324+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb30102d90 0x7fcb30104160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00d740 tx=0x7fcb1c00da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.325+0000 7fcb27400700 1 -- 172.21.15.37:0/3252743659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c017070 con 0x7fcb30102d90 2024-04-26T02:00:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.325+0000 7fcb27400700 1 -- 172.21.15.37:0/3252743659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1c00f690 con 0x7fcb30102d90 2024-04-26T02:00:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.325+0000 7fcb27400700 1 -- 172.21.15.37:0/3252743659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c003b40 con 0x7fcb30102d90 2024-04-26T02:00:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.326+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb3019b430 con 0x7fcb30102d90 2024-04-26T02:00:30.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.326+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb3019b920 con 0x7fcb30102d90 2024-04-26T02:00:30.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.334+0000 7fcb27400700 1 -- 172.21.15.37:0/3252743659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcb1c00f840 con 0x7fcb30102d90 2024-04-26T02:00:30.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.336+0000 7fcb40024b80 1 Processor -- start 2024-04-26T02:00:30.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.336+0000 7fcb40024b80 1 -- start start 2024-04-26T02:00:30.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.337+0000 7fcb27400700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fcb140a3ed0 0x7fcb140a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.337+0000 7fcb27400700 1 -- 172.21.15.37:0/3252743659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fcb1c0cfbe0 con 0x7fcb30102d90 2024-04-26T02:00:30.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.338+0000 7fcb2f400700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fcb140a3ed0 0x7fcb140a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.338+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 0x5631da6a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.338+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.338+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb40024b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5631da5f9020 con 0x5631da6b1560 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb40024b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5631da59f8c0 con 0x5631da6a66f0 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb40024b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5631da5f4fe0 con 0x5631da6a5dd0 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2f400700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fcb140a3ed0 0x7fcb140a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009550 tx=0x7fcb28009040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 0x5631da6a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2e000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 0x5631da6a61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43014/0 (socket says 172.21.15.37:43014) 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2ea00700 1 -- 172.21.15.37:0/3507150675 learned_addr learned my addr 172.21.15.37:0/3507150675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2f400700 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2e000700 1 -- 172.21.15.37:0/3507150675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 msgr2=0x5631da6a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 0x5631da6a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2e000700 1 -- 172.21.15.37:0/3507150675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 msgr2=0x5631da6b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2e000700 1 -- 172.21.15.37:0/3507150675 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631da595810 con 0x5631da6a66f0 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.339+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 0x5631da6a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.340+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6b1020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0cd300 tx=0x7fcb1c09b5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f868348324b8a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.340+0000 7fcb1be00700 1 -- 172.21.15.37:0/3507150675 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c020040 con 0x5631da6a66f0 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.340+0000 7fcb1be00700 1 -- 172.21.15.37:0/3507150675 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1c09ff00 con 0x5631da6a66f0 2024-04-26T02:00:30.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.340+0000 7fcb1be00700 1 -- 172.21.15.37:0/3507150675 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb1c09bdb0 con 0x5631da6a66f0 2024-04-26T02:00:30.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 -- 172.21.15.37:0/3507150675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 msgr2=0x5631da6b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6b1020 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c0cd300 tx=0x7fcb1c09b5d0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 -- 172.21.15.37:0/3507150675 shutdown_connections 2024-04-26T02:00:30.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a5dd0 0x5631da6a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3507150675 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6b1020 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 -- 172.21.15.37:0/3507150675 >> 172.21.15.37:0/3507150675 conn(0x5631da5d50e0 msgr2=0x5631da6a22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 -- 172.21.15.37:0/3507150675 shutdown_connections 2024-04-26T02:00:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.341+0000 7fcb40024b80 1 -- 172.21.15.37:0/3507150675 wait complete. 2024-04-26T02:00:30.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.342+0000 7fcb40024b80 1 -- start start 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.342+0000 7fcb40024b80 1 Processor -- start 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 0x5631da6b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6ae080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6b1560 0x5631da6ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5631da6b3f30 con 0x5631da6a5dd0 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5631da5b29f0 con 0x5631da6a66f0 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5631da6b40b0 con 0x5631da6b1560 2024-04-26T02:00:30.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.343+0000 7fcb40024b80 10 client.0 _ll_register_callbacks cb 0x5631da5918b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.344+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 0x5631da6b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.344+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 0x5631da6b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34016/0 (socket says 172.21.15.37:34016) 2024-04-26T02:00:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.344+0000 7fcb2ea00700 1 -- 172.21.15.37:0/1080371747 learned_addr learned my addr 172.21.15.37:0/1080371747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.344+0000 7fcb2f400700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6b1560 0x5631da6ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2ea00700 1 -- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6b1560 msgr2=0x5631da6ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6b1560 0x5631da6ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2ea00700 1 -- 172.21.15.37:0/1080371747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 msgr2=0x5631da6ae080 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6ae080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2ea00700 1 -- 172.21.15.37:0/1080371747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631da546a60 con 0x5631da6a5dd0 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2f400700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6b1560 0x5631da6ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 0x5631da6b0d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000bf90 tx=0x7fcb2000bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c0facbc0224db1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:30.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.345+0000 7fcb18c00700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:00:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.346+0000 7fcb18c00700 1 -- 172.21.15.37:0/1080371747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20003a60 con 0x5631da6a5dd0 2024-04-26T02:00:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.346+0000 7fcb18c00700 1 -- 172.21.15.37:0/1080371747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fcb20003c00 con 0x5631da6a5dd0 2024-04-26T02:00:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.346+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb1c09ff00 con 0x5631da6a5dd0 2024-04-26T02:00:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.347+0000 7fcb18c00700 1 -- 172.21.15.37:0/1080371747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcb20003e00 con 0x5631da6a5dd0 2024-04-26T02:00:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.347+0000 7fcb40024b80 10 client.15243 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:30.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.347+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5631da58fc80 con 0x5631da6a5dd0 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb18c00700 1 -- 172.21.15.37:0/1080371747 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb1c09ff00 con 0x5631da6a5dd0 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb18c00700 1 -- 172.21.15.37:0/1080371747 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fcb200238e0 con 0x5631da6a5dd0 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb40024b80 10 client.15243 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb40024b80 10 client.15243 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb40024b80 20 client.15243 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb40024b80 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da6c10e0 0x5631da6c3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb40024b80 4 client.15243 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631da606810 con 0x5631da6c10e0 2024-04-26T02:00:30.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.348+0000 7fcb2e000700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da6c10e0 0x5631da6c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.349+0000 7fcb18c00700 10 client.15243 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:30.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.349+0000 7fcb2e000700 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da6c10e0 0x5631da6c3540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=114ec0534d2ea60b server_cookie=55bbe21ced0591ec in_seq=0 out_seq=0 2024-04-26T02:00:30.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.350+0000 7fcb18c00700 1 -- 172.21.15.37:0/1080371747 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631da606810 con 0x5631da6c10e0 2024-04-26T02:00:30.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.350+0000 7fcb18c00700 10 client.15243 handle_command_reply: tid=0 2024-04-26T02:00:30.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.350+0000 7fcb40024b80 1 client.15243 shutdown 2024-04-26T02:00:30.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.350+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 msgr2=0x5631da6b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.350+0000 7fcb40024b80 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 0x5631da6b0d80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000bf90 tx=0x7fcb2000bfc0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 shutdown_connections 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da6c10e0 0x5631da6c3540 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6a5dd0 0x5631da6b0d80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6b1560 0x5631da6ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 --2- 172.21.15.37:0/1080371747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6a66f0 0x5631da6ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 >> 172.21.15.37:0/1080371747 conn(0x5631da5d50e0 msgr2=0x5631da6a1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb18c00700 0 client.15243 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 shutdown_connections 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 1 -- 172.21.15.37:0/1080371747 wait complete. 2024-04-26T02:00:30.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.351+0000 7fcb40024b80 20 client.15243 trim_cache size 0 max 16384 2024-04-26T02:00:30.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.396+0000 7fcb40024b80 1 Processor -- start 2024-04-26T02:00:30.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 -- start start 2024-04-26T02:00:30.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6a2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6bcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da6b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301a1150 con 0x5631da6b1560 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301bcae0 con 0x5631da6b9300 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb40024b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301a17c0 con 0x5631da6a66f0 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43030/0 (socket says 172.21.15.37:43030) 2024-04-26T02:00:30.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.397+0000 7fcb2ea00700 1 -- 172.21.15.37:0/3159238963 learned_addr learned my addr 172.21.15.37:0/3159238963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6bcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2ea00700 1 -- 172.21.15.37:0/3159238963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 msgr2=0x5631da6b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2f400700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da6b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da6b9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2ea00700 1 -- 172.21.15.37:0/3159238963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 msgr2=0x5631da6bcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6bcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2ea00700 1 -- 172.21.15.37:0/3159238963 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631da6a6d70 con 0x5631da6a66f0 2024-04-26T02:00:30.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2e000700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:30.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.398+0000 7fcb2f400700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da6b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:30.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.399+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6a2430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb200098e0 tx=0x7fcb2000f460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83ad8ebbe037874f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:30.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.399+0000 7fcb1b400700 1 -- 172.21.15.37:0/3159238963 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2003b070 con 0x5631da6a66f0 2024-04-26T02:00:30.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.399+0000 7fcb1b400700 1 -- 172.21.15.37:0/3159238963 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb20028070 con 0x5631da6a66f0 2024-04-26T02:00:30.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 -- 172.21.15.37:0/3159238963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 msgr2=0x5631da6a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6a2430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb200098e0 tx=0x7fcb2000f460 comp rx=0 tx=0).stop 2024-04-26T02:00:30.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.399+0000 7fcb1b400700 1 -- 172.21.15.37:0/3159238963 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb20005040 con 0x5631da6a66f0 2024-04-26T02:00:30.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 -- 172.21.15.37:0/3159238963 shutdown_connections 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da6b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6a2430 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 --2- 172.21.15.37:0/3159238963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6bcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 -- 172.21.15.37:0/3159238963 >> 172.21.15.37:0/3159238963 conn(0x5631da5d50e0 msgr2=0x5631da6a1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.400+0000 7fcb40024b80 1 -- 172.21.15.37:0/3159238963 shutdown_connections 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.401+0000 7fcb40024b80 1 -- 172.21.15.37:0/3159238963 wait complete. 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.401+0000 7fcb40024b80 1 -- start start 2024-04-26T02:00:30.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.401+0000 7fcb40024b80 1 Processor -- start 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6b7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da7276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301a1e90 con 0x5631da6b1560 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301a3300 con 0x5631da6b9300 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fcb301a2560 con 0x5631da6a66f0 2024-04-26T02:00:30.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.402+0000 7fcb40024b80 10 client.0 _ll_register_callbacks cb 0x5631da5918b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:30.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da7276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da7276a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:42470/0 (socket says 172.21.15.37:42470) 2024-04-26T02:00:30.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2f400700 1 -- 172.21.15.37:0/2613614543 learned_addr learned my addr 172.21.15.37:0/2613614543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6b7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2e000700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 -- 172.21.15.37:0/2613614543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 msgr2=0x5631da7276a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da7276a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 -- 172.21.15.37:0/2613614543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 msgr2=0x5631da6b7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 -- 172.21.15.37:0/2613614543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631da727be0 con 0x5631da6a66f0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2e000700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb2ea00700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6b7600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000f4f0 tx=0x7fcb200238d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4752fa5c141bc90 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.403+0000 7fcb18c00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.404+0000 7fcb18c00700 1 -- 172.21.15.37:0/2613614543 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb2003b070 con 0x5631da6a66f0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.404+0000 7fcb18c00700 1 -- 172.21.15.37:0/2613614543 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fcb20005070 con 0x5631da6a66f0 2024-04-26T02:00:30.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.404+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb20028070 con 0x5631da6a66f0 2024-04-26T02:00:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.404+0000 7fcb18c00700 1 -- 172.21.15.37:0/2613614543 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fcb20045050 con 0x5631da6a66f0 2024-04-26T02:00:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.405+0000 7fcb40024b80 10 client.25393 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.405+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5631da728d00 con 0x5631da6a66f0 2024-04-26T02:00:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.405+0000 7fcb18c00700 1 -- 172.21.15.37:0/2613614543 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb20028070 con 0x5631da6a66f0 2024-04-26T02:00:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb18c00700 1 -- 172.21.15.37:0/2613614543 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fcb20035070 con 0x5631da6a66f0 2024-04-26T02:00:30.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb40024b80 10 client.25393 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb40024b80 10 client.25393 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:00:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb40024b80 20 client.25393 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb40024b80 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da729360 0x5631da72b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb40024b80 4 client.25393 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T02:00:30.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.406+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5631da6008b0 con 0x5631da729360 2024-04-26T02:00:30.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.408+0000 7fcb2e000700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da729360 0x5631da72b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:30.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.409+0000 7fcb2e000700 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da729360 0x5631da72b750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de195d622e21da48 server_cookie=2d207040a4d225bb in_seq=0 out_seq=0 2024-04-26T02:00:30.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.409+0000 7fcb18c00700 10 client.25393 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:30.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb18c00700 1 -- 172.21.15.37:0/2613614543 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5631da6008b0 con 0x5631da729360 2024-04-26T02:00:30.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb18c00700 10 client.25393 handle_command_reply: tid=0 2024-04-26T02:00:30.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 client.25393 shutdown 2024-04-26T02:00:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 msgr2=0x5631da6b7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6b7600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb2000f4f0 tx=0x7fcb200238d0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 shutdown_connections 2024-04-26T02:00:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5631da729360 0x5631da72b750 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5631da6b9300 0x5631da7276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5631da6a66f0 0x5631da6b7600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb18c00700 0 client.25393 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 --2- 172.21.15.37:0/2613614543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631da6b1560 0x5631da6b7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.420+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 >> 172.21.15.37:0/2613614543 conn(0x5631da5d50e0 msgr2=0x5631da6bbde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.421+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 shutdown_connections 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.421+0000 7fcb40024b80 1 -- 172.21.15.37:0/2613614543 wait complete. 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.421+0000 7fcb40024b80 20 client.25393 trim_cache size 0 max 16384 2024-04-26T02:00:30.420 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 6, 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "cf94f552": { 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 6, 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:00:30.421 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:00:30.422 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:00:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fcb140a3ed0 msgr2=0x7fcb140a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fcb140a3ed0 0x7fcb140a6390 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcb28009550 tx=0x7fcb28009040 comp rx=0 tx=0).stop 2024-04-26T02:00:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb30102d90 msgr2=0x7fcb30104160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb30102d90 0x7fcb30104160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcb1c00d740 tx=0x7fcb1c00da50 comp rx=0 tx=0).stop 2024-04-26T02:00:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 shutdown_connections 2024-04-26T02:00:30.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fcb140a3ed0 0x7fcb140a6390 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fcb3010f4f0 0x7fcb301046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.422+0000 7fcb35600700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fcb30102470 0x7fcb30103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.423+0000 7fcb35600700 1 --2- 172.21.15.37:0/3252743659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb30102d90 0x7fcb30104160 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.423+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 >> 172.21.15.37:0/3252743659 conn(0x7fcb300788e0 msgr2=0x7fcb30110900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.423+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 shutdown_connections 2024-04-26T02:00:30.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:00:30.423+0000 7fcb35600700 1 -- 172.21.15.37:0/3252743659 wait complete. 2024-04-26T02:00:31.642 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cf94f552', quiesce took 11.0 seconds. Will release after: 30.0 2024-04-26T02:00:31.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:31 smithi037 ceph-mon[27906]: pgmap v80: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 242 MiB/s wr, 90 op/s 2024-04-26T02:00:32.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:31 smithi071 ceph-mon[32578]: pgmap v80: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 242 MiB/s wr, 90 op/s 2024-04-26T02:00:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:31 smithi045 ceph-mon[32520]: pgmap v80: 97 pgs: 97 active+clean; 3.8 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 242 MiB/s wr, 90 op/s 2024-04-26T02:00:34.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:33 smithi071 ceph-mon[32578]: pgmap v81: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 403 MiB/s wr, 133 op/s 2024-04-26T02:00:34.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:33 smithi037 ceph-mon[27906]: pgmap v81: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 403 MiB/s wr, 133 op/s 2024-04-26T02:00:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:33 smithi045 ceph-mon[32520]: pgmap v81: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 403 MiB/s wr, 133 op/s 2024-04-26T02:00:36.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:35 smithi071 ceph-mon[32578]: pgmap v82: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 83 op/s 2024-04-26T02:00:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:35 smithi037 ceph-mon[27906]: pgmap v82: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 83 op/s 2024-04-26T02:00:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:35 smithi045 ceph-mon[32520]: pgmap v82: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 83 op/s 2024-04-26T02:00:38.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:37 smithi071 ceph-mon[32578]: pgmap v83: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 83 op/s 2024-04-26T02:00:38.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 -- 172.21.15.37:0/3316817157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4748007630 msgr2=0x7f4748007a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 --2- 172.21.15.37:0/3316817157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4748007630 0x7f4748007a10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4744009900 tx=0x7f474402a420 comp rx=0 tx=0).stop 2024-04-26T02:00:38.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:37 smithi037 ceph-mon[27906]: pgmap v83: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 83 op/s 2024-04-26T02:00:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 -- 172.21.15.37:0/3316817157 shutdown_connections 2024-04-26T02:00:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 --2- 172.21.15.37:0/3316817157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f47480a2910 0x7f47480a4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 --2- 172.21.15.37:0/3316817157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007f50 0x7f47480a23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 --2- 172.21.15.37:0/3316817157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4748007630 0x7f4748007a10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.209+0000 7f4755400640 1 -- 172.21.15.37:0/3316817157 >> 172.21.15.37:0/3316817157 conn(0x7f474800b8c0 msgr2=0x7f474800bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:38.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.210+0000 7f4755400640 1 -- 172.21.15.37:0/3316817157 shutdown_connections 2024-04-26T02:00:38.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.212+0000 7f4755400640 1 -- 172.21.15.37:0/3316817157 wait complete. 2024-04-26T02:00:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.213+0000 7f4755400640 1 Processor -- start 2024-04-26T02:00:38.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.213+0000 7f4755400640 1 -- start start 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.214+0000 7f4755400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 0x7f4748134410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.214+0000 7f4755400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4748007f50 0x7f4748134950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.214+0000 7f4755400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47480a2910 0x7f474812e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.214+0000 7f4755400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47480a81c0 con 0x7f4748007630 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.214+0000 7f4755400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47480a8040 con 0x7f4748007f50 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.214+0000 7f4755400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f47480a8340 con 0x7f47480a2910 2024-04-26T02:00:38.213 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.215+0000 7f474fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 0x7f4748134410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.215+0000 7f474fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 0x7f4748134410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34040/0 (socket says 172.21.15.37:34040) 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.215+0000 7f474fe00640 1 -- 172.21.15.37:0/2458839791 learned_addr learned my addr 172.21.15.37:0/2458839791 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.216+0000 7f474fe00640 1 -- 172.21.15.37:0/2458839791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47480a2910 msgr2=0x7f474812e5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.216+0000 7f474fe00640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47480a2910 0x7f474812e5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.216+0000 7f474fe00640 1 -- 172.21.15.37:0/2458839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4748007f50 msgr2=0x7f4748134950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.216+0000 7f474fe00640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4748007f50 0x7f4748134950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.216+0000 7f474fe00640 1 -- 172.21.15.37:0/2458839791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474812ed40 con 0x7f4748007630 2024-04-26T02:00:38.215 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.216+0000 7f474fe00640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 0x7f4748134410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47440316d0 tx=0x7f4744031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.218+0000 7f474cc00640 1 -- 172.21.15.37:0/2458839791 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474403f030 con 0x7f4748007630 2024-04-26T02:00:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.218+0000 7f474cc00640 1 -- 172.21.15.37:0/2458839791 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4744031920 con 0x7f4748007630 2024-04-26T02:00:38.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.218+0000 7f474cc00640 1 -- 172.21.15.37:0/2458839791 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4744035070 con 0x7f4748007630 2024-04-26T02:00:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.218+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474812efd0 con 0x7f4748007630 2024-04-26T02:00:38.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.218+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4748095f60 con 0x7f4748007630 2024-04-26T02:00:38.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.224+0000 7f474cc00640 1 -- 172.21.15.37:0/2458839791 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4744042030 con 0x7f4748007630 2024-04-26T02:00:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.224+0000 7f4759800740 1 Processor -- start 2024-04-26T02:00:38.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.225+0000 7f4759800740 1 -- start start 2024-04-26T02:00:38.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.226+0000 7f4759800740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 0x55905d62dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.226+0000 7f4759800740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62e2d0 0x55905d638650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.226+0000 7f4754a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 0x55905d62dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.227+0000 7f4754a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 0x55905d62dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43058/0 (socket says 172.21.15.37:43058) 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.227+0000 7f4754a00640 1 -- 172.21.15.37:0/590122402 learned_addr learned my addr 172.21.15.37:0/590122402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.227+0000 7f4759800740 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.227+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55905d593900 con 0x55905d62e2d0 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.227+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55905d539ef0 con 0x55905d638b90 2024-04-26T02:00:38.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.227+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55905d58f8c0 con 0x55905d62d9b0 2024-04-26T02:00:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f474cc00640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f47380a3e20 0x7f47380a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f474f400640 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f474fe00640 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62e2d0 0x55905d638650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f474cc00640 1 -- 172.21.15.37:0/2458839791 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f47440f0cb0 con 0x7f4748007630 2024-04-26T02:00:38.226 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f4754a00640 1 -- 172.21.15.37:0/590122402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 msgr2=0x55905d63af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f4754a00640 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f4754a00640 1 -- 172.21.15.37:0/590122402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62e2d0 msgr2=0x55905d638650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f4754a00640 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62e2d0 0x55905d638650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.228+0000 7f4754a00640 1 -- 172.21.15.37:0/590122402 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55905d2a6210 con 0x55905d62d9b0 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.229+0000 7f474fe00640 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62e2d0 0x55905d638650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.229+0000 7f4754a00640 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 0x55905d62dd90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4750050a90 tx=0x7f475006aff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7128069431191ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:38.227 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.229+0000 7f4754a00640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f47380a3e20 0x7f47380a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.229+0000 7f4741200640 1 -- 172.21.15.37:0/590122402 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f475008a880 con 0x55905d62d9b0 2024-04-26T02:00:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.230+0000 7f4741200640 1 -- 172.21.15.37:0/590122402 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750075070 con 0x55905d62d9b0 2024-04-26T02:00:38.228 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.230+0000 7f4741200640 1 -- 172.21.15.37:0/590122402 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4750089ad0 con 0x55905d62d9b0 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.230+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 msgr2=0x55905d62dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.230+0000 7f4759800740 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 0x55905d62dd90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4750050a90 tx=0x7f475006aff0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4754a00640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f47380a3e20 0x7f47380a62e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4750050760 tx=0x7f475008c000 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 shutdown_connections 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62e2d0 0x55905d638650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 --2- 172.21.15.37:0/590122402 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62d9b0 0x55905d62dd90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 >> 172.21.15.37:0/590122402 conn(0x55905d629ec0 msgr2=0x55905d62a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:38.229 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 shutdown_connections 2024-04-26T02:00:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 -- 172.21.15.37:0/590122402 wait complete. 2024-04-26T02:00:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.231+0000 7f4759800740 1 -- start start 2024-04-26T02:00:38.230 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.232+0000 7f4759800740 1 Processor -- start 2024-04-26T02:00:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.232+0000 7f4759800740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d635b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.233+0000 7f4759800740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 0x55905d636050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.233+0000 7f4759800740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d638b90 0x55905d636590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.231 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.233+0000 7f474fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d635b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.233+0000 7f474fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d635b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34056/0 (socket says 172.21.15.37:34056) 2024-04-26T02:00:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f474f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 0x55905d636050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.232 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f474f400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 0x55905d636050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:42508/0 (socket says 172.21.15.37:42508) 2024-04-26T02:00:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f474fe00640 1 -- 172.21.15.37:0/780904701 learned_addr learned my addr 172.21.15.37:0/780904701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f4754a00640 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d638b90 0x55905d636590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55905d63b7c0 con 0x55905d62d9b0 2024-04-26T02:00:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55905d54d310 con 0x55905d62e2d0 2024-04-26T02:00:38.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.234+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55905d63b940 con 0x55905d638b90 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f474f400640 1 -- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d638b90 msgr2=0x55905d636590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f474f400640 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d638b90 0x55905d636590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f474f400640 1 -- 172.21.15.37:0/780904701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 msgr2=0x55905d635b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f474f400640 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d635b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f474f400640 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905d372620 con 0x55905d62e2d0 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f4759800740 10 client.0 _ll_register_callbacks cb 0x55905d51bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:38.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.235+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4750075070 con 0x55905d62e2d0 2024-04-26T02:00:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.238+0000 7f474f400640 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 0x55905d636050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55905d62b2e0 tx=0x7f473c00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=104feb281c6fc3f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:38.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.238+0000 7f4735600640 10 client.25346 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:00:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.244+0000 7f4735600640 1 -- 172.21.15.37:0/780904701 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c015070 con 0x55905d62e2d0 2024-04-26T02:00:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.245+0000 7f4735600640 1 -- 172.21.15.37:0/780904701 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f473c004220 con 0x55905d62e2d0 2024-04-26T02:00:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.245+0000 7f4735600640 1 -- 172.21.15.37:0/780904701 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f473c004460 con 0x55905d62e2d0 2024-04-26T02:00:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.245+0000 7f4759800740 10 client.25346 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:38.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.245+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({fsmap=19}) -- 0x55905d647800 con 0x55905d62e2d0 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4735600640 1 -- 172.21.15.37:0/780904701 <== mon.2 v2:172.21.15.45:3300/0 4 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f473c0035f0 con 0x55905d62e2d0 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4759800740 10 client.25346 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4759800740 10 client.25346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4759800740 20 client.25346 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4759800740 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d649000 0x55905d64b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4759800740 4 client.25346 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:38.244 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55905d5a78b0 con 0x55905d649000 2024-04-26T02:00:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.246+0000 7f474fe00640 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d649000 0x55905d64b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.247+0000 7f474fe00640 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d649000 0x55905d64b4c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61bacc9456b08b45 server_cookie=3cd6bf05f9d40b70 in_seq=0 out_seq=0 2024-04-26T02:00:38.245 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.247+0000 7f4735600640 10 client.25346 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.248+0000 7f4735600640 1 -- 172.21.15.37:0/780904701 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55905d5a78b0 con 0x55905d649000 2024-04-26T02:00:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.248+0000 7f4735600640 10 client.25346 handle_command_reply: tid=0 2024-04-26T02:00:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 client.25346 shutdown 2024-04-26T02:00:38.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 msgr2=0x55905d636050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 0x55905d636050 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x55905d62b2e0 tx=0x7f473c00f730 comp rx=0 tx=0).stop 2024-04-26T02:00:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 shutdown_connections 2024-04-26T02:00:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d649000 0x55905d64b4c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d638b90 0x55905d636590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d62e2d0 0x55905d636050 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 --2- 172.21.15.37:0/780904701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d635b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 >> 172.21.15.37:0/780904701 conn(0x55905d629ec0 msgr2=0x55905d62fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.249+0000 7f4735600640 0 client.25346 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.250+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 shutdown_connections 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.250+0000 7f4759800740 1 -- 172.21.15.37:0/780904701 wait complete. 2024-04-26T02:00:38.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.250+0000 7f4759800740 20 client.25346 trim_cache size 0 max 16384 2024-04-26T02:00:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:37 smithi045 ceph-mon[32520]: pgmap v83: 97 pgs: 97 active+clean; 5.7 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 301 MiB/s wr, 83 op/s 2024-04-26T02:00:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 Processor -- start 2024-04-26T02:00:38.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 -- start start 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d630370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d6363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d637a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4748146030 con 0x55905d62d9b0 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4748148870 con 0x55905d638b90 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.272+0000 7f4759800740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4748146680 con 0x55905d62e2d0 2024-04-26T02:00:38.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d6363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d6363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43070/0 (socket says 172.21.15.37:43070) 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 -- 172.21.15.37:0/2509144687 learned_addr learned my addr 172.21.15.37:0/2509144687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f4754a00640 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d637a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 -- 172.21.15.37:0/2509144687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 msgr2=0x55905d637a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d637a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 -- 172.21.15.37:0/2509144687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 msgr2=0x55905d630370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d630370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 -- 172.21.15.37:0/2509144687 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55905d645470 con 0x55905d62e2d0 2024-04-26T02:00:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.273+0000 7f474f400640 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d6363b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f473c010010 tx=0x7f473c009000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfdd20e91dc5d1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.274+0000 7f4737e00640 1 -- 172.21.15.37:0/2509144687 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c032070 con 0x55905d62e2d0 2024-04-26T02:00:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.274+0000 7f4737e00640 1 -- 172.21.15.37:0/2509144687 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750075070 con 0x55905d62e2d0 2024-04-26T02:00:38.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.274+0000 7f4737e00640 1 -- 172.21.15.37:0/2509144687 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c01a6c0 con 0x55905d62e2d0 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.274+0000 7f4759800740 1 -- 172.21.15.37:0/2509144687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 msgr2=0x55905d6363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.274+0000 7f4759800740 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d6363b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f473c010010 tx=0x7f473c009000 comp rx=0 tx=0).stop 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.275+0000 7f4759800740 1 -- 172.21.15.37:0/2509144687 shutdown_connections 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.275+0000 7f4759800740 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d637a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.275+0000 7f4759800740 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d6363b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.275+0000 7f4759800740 1 --2- 172.21.15.37:0/2509144687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d630370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.275+0000 7f4759800740 1 -- 172.21.15.37:0/2509144687 >> 172.21.15.37:0/2509144687 conn(0x55905d629ec0 msgr2=0x55905d637340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.278+0000 7f4759800740 1 -- 172.21.15.37:0/2509144687 shutdown_connections 2024-04-26T02:00:38.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.278+0000 7f4759800740 1 -- 172.21.15.37:0/2509144687 wait complete. 2024-04-26T02:00:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 -- start start 2024-04-26T02:00:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 Processor -- start 2024-04-26T02:00:38.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d640d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d63e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4748146d30 con 0x55905d62d9b0 2024-04-26T02:00:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f47481481a0 con 0x55905d638b90 2024-04-26T02:00:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.279+0000 7f4759800740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4748147400 con 0x55905d62e2d0 2024-04-26T02:00:38.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.280+0000 7f4759800740 10 client.0 _ll_register_callbacks cb 0x7f473c00b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.280+0000 7f474f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d63e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.280+0000 7f4754a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.280+0000 7f474f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d63e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:43074/0 (socket says 172.21.15.37:43074) 2024-04-26T02:00:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.280+0000 7f474f400640 1 -- 172.21.15.37:0/3835539022 learned_addr learned my addr 172.21.15.37:0/3835539022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:38.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f474f400640 1 -- 172.21.15.37:0/3835539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 msgr2=0x55905d63e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f474f400640 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f474f400640 1 -- 172.21.15.37:0/3835539022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 msgr2=0x55905d640d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f474f400640 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d640d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f474f400640 1 -- 172.21.15.37:0/3835539022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55905d63eca0 con 0x55905d62e2d0 2024-04-26T02:00:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f474f400640 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d63e120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b950 tx=0x7f473c003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5f9e0d2184f729 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:38.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.281+0000 7f4735600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:00:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.282+0000 7f4735600640 1 -- 172.21.15.37:0/3835539022 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c03b060 con 0x55905d62e2d0 2024-04-26T02:00:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.282+0000 7f4735600640 1 -- 172.21.15.37:0/3835539022 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f473c004100 con 0x55905d62e2d0 2024-04-26T02:00:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.282+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4750075070 con 0x55905d62e2d0 2024-04-26T02:00:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.282+0000 7f4735600640 1 -- 172.21.15.37:0/3835539022 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f473c015070 con 0x55905d62e2d0 2024-04-26T02:00:38.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.282+0000 7f4759800740 10 client.25408 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.282+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55905d6af490 con 0x55905d62e2d0 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4735600640 1 -- 172.21.15.37:0/3835539022 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4750075070 con 0x55905d62e2d0 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4735600640 1 -- 172.21.15.37:0/3835539022 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f473c01a460 con 0x55905d62e2d0 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4759800740 10 client.25408 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4759800740 10 client.25408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4759800740 20 client.25408 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4759800740 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d6afb50 0x55905d6b1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4759800740 4 client.25408 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:38.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.283+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55905d5a1320 con 0x55905d6afb50 2024-04-26T02:00:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.284+0000 7f474fe00640 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d6afb50 0x55905d6b1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:38.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.285+0000 7f474fe00640 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d6afb50 0x55905d6b1f70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523a600aa4067d7a server_cookie=7659fd5dc2ac18d1 in_seq=0 out_seq=0 2024-04-26T02:00:38.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.285+0000 7f4735600640 10 client.25408 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.308+0000 7f4735600640 1 -- 172.21.15.37:0/3835539022 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55905d5a1320 con 0x55905d6afb50 2024-04-26T02:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.308+0000 7f4735600640 10 client.25408 handle_command_reply: tid=0 2024-04-26T02:00:38.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.308+0000 7f4759800740 1 client.25408 shutdown 2024-04-26T02:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.308+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 msgr2=0x55905d63e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d63e120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b950 tx=0x7f473c003f90 comp rx=0 tx=0).stop 2024-04-26T02:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 shutdown_connections 2024-04-26T02:00:38.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55905d6afb50 0x55905d6b1f70 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55905d638b90 0x55905d63e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55905d62e2d0 0x55905d63e120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 --2- 172.21.15.37:0/3835539022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55905d62d9b0 0x55905d640d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 >> 172.21.15.37:0/3835539022 conn(0x55905d629ec0 msgr2=0x55905d637bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.309+0000 7f4735600640 0 client.25408 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.310+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 shutdown_connections 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.310+0000 7f4759800740 1 -- 172.21.15.37:0/3835539022 wait complete. 2024-04-26T02:00:38.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.310+0000 7f4759800740 20 client.25408 trim_cache size 0 max 16384 2024-04-26T02:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:00:38.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.311+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f47380a3e20 msgr2=0x7f47380a62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.311+0000 7f4755400640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f47380a3e20 0x7f47380a62e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4750050760 tx=0x7f475008c000 comp rx=0 tx=0).stop 2024-04-26T02:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.311+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 msgr2=0x7f4748134410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:38.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.311+0000 7f4755400640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 0x7f4748134410 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f47440316d0 tx=0x7f4744031700 comp rx=0 tx=0).stop 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.312+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 shutdown_connections 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.312+0000 7f4755400640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f47380a3e20 0x7f47380a62e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.312+0000 7f4755400640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f47480a2910 0x7f474812e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.312+0000 7f4755400640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4748007f50 0x7f4748134950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.312+0000 7f4755400640 1 --2- 172.21.15.37:0/2458839791 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4748007630 0x7f4748134410 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.312+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 >> 172.21.15.37:0/2458839791 conn(0x7f474800b8c0 msgr2=0x7f47480931c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.313+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 shutdown_connections 2024-04-26T02:00:38.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:38.313+0000 7f4755400640 1 -- 172.21.15.37:0/2458839791 wait complete. 2024-04-26T02:00:40.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:39 smithi037 ceph-mon[27906]: pgmap v84: 97 pgs: 97 active+clean; 7.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 475 MiB/s wr, 127 op/s 2024-04-26T02:00:40.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:39 smithi045 ceph-mon[32520]: pgmap v84: 97 pgs: 97 active+clean; 7.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 475 MiB/s wr, 127 op/s 2024-04-26T02:00:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:39 smithi071 ceph-mon[32578]: pgmap v84: 97 pgs: 97 active+clean; 7.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 475 MiB/s wr, 127 op/s 2024-04-26T02:00:42.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:41 smithi037 ceph-mon[27906]: pgmap v85: 97 pgs: 97 active+clean; 7.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 334 MiB/s wr, 86 op/s 2024-04-26T02:00:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:41 smithi045 ceph-mon[32520]: pgmap v85: 97 pgs: 97 active+clean; 7.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 334 MiB/s wr, 86 op/s 2024-04-26T02:00:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:41 smithi071 ceph-mon[32578]: pgmap v85: 97 pgs: 97 active+clean; 7.8 GiB data, 23 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 334 MiB/s wr, 86 op/s 2024-04-26T02:00:44.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:43 smithi037 ceph-mon[27906]: pgmap v86: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 460 MiB/s wr, 121 op/s 2024-04-26T02:00:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:43 smithi045 ceph-mon[32520]: pgmap v86: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 460 MiB/s wr, 121 op/s 2024-04-26T02:00:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:43 smithi071 ceph-mon[32578]: pgmap v86: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 460 MiB/s wr, 121 op/s 2024-04-26T02:00:46.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:45 smithi071 ceph-mon[32578]: pgmap v87: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 299 MiB/s wr, 79 op/s 2024-04-26T02:00:46.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:45 smithi037 ceph-mon[27906]: pgmap v87: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 299 MiB/s wr, 79 op/s 2024-04-26T02:00:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:45 smithi045 ceph-mon[32520]: pgmap v87: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 299 MiB/s wr, 79 op/s 2024-04-26T02:00:47.958 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:47 smithi037 ceph-mon[27906]: pgmap v88: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 299 MiB/s wr, 79 op/s 2024-04-26T02:00:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:47 smithi045 ceph-mon[32520]: pgmap v88: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 299 MiB/s wr, 79 op/s 2024-04-26T02:00:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:47 smithi071 ceph-mon[32578]: pgmap v88: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 299 MiB/s wr, 79 op/s 2024-04-26T02:00:50.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:49 smithi037 ceph-mon[27906]: pgmap v89: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 299 MiB/s wr, 80 op/s 2024-04-26T02:00:50.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:49 smithi045 ceph-mon[32520]: pgmap v89: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 299 MiB/s wr, 80 op/s 2024-04-26T02:00:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:49 smithi071 ceph-mon[32578]: pgmap v89: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 299 MiB/s wr, 80 op/s 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1924970979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 msgr2=0x7f7bf010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1924970979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 0x7f7bf010e7f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009be0 tx=0x7f7be802a750 comp rx=0 tx=0).stop 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1924970979 shutdown_connections 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1924970979 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 0x7f7bf010e7f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1924970979 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bf0103850 0x7f7bf010bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1924970979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0102f30 0x7f7bf0103310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1924970979 >> 172.21.15.37:0/1924970979 conn(0x7f7bf00fc730 msgr2=0x7f7bf00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1924970979 shutdown_connections 2024-04-26T02:00:51.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.468+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1924970979 wait complete. 2024-04-26T02:00:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.469+0000 7f7bf4a00640 1 Processor -- start 2024-04-26T02:00:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 -- start start 2024-04-26T02:00:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bf0102f30 0x7f7bf019a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 0x7f7bf019acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 0x7f7bf019f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0111fe0 con 0x7f7bf0103850 2024-04-26T02:00:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0111e60 con 0x7f7bf0102f30 2024-04-26T02:00:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bf4a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf0112160 con 0x7f7bf010c350 2024-04-26T02:00:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bede00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 0x7f7bf019acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bede00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 0x7f7bf019acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45152/0 (socket says 172.21.15.37:45152) 2024-04-26T02:00:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bede00640 1 -- 172.21.15.37:0/1055971398 learned_addr learned my addr 172.21.15.37:0/1055971398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.470+0000 7f7bef200640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 0x7f7bf019f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bee800640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bf0102f30 0x7f7bf019a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bede00640 1 -- 172.21.15.37:0/1055971398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 msgr2=0x7f7bf019f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bede00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 0x7f7bf019f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bede00640 1 -- 172.21.15.37:0/1055971398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bf0102f30 msgr2=0x7f7bf019a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bede00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bf0102f30 0x7f7bf019a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bede00640 1 -- 172.21.15.37:0/1055971398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf019f730 con 0x7f7bf0103850 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7bede00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 0x7f7bf019acd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e9f0 tx=0x7f7bdc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7be7400640 1 -- 172.21.15.37:0/1055971398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc00cab0 con 0x7f7bf0103850 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.471+0000 7f7be7400640 1 -- 172.21.15.37:0/1055971398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bdc0054d0 con 0x7f7bf0103850 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.472+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf019fa20 con 0x7f7bf0103850 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.472+0000 7f7be7400640 1 -- 172.21.15.37:0/1055971398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bdc004e40 con 0x7f7bf0103850 2024-04-26T02:00:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.472+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf01a72b0 con 0x7f7bf0103850 2024-04-26T02:00:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.475+0000 7f7bf822a740 1 Processor -- start 2024-04-26T02:00:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.475+0000 7f7bf822a740 1 -- start start 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d21d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d2c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bf822a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a18c87900 con 0x562a18d219b0 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bf822a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a18c2def0 con 0x562a18d2cb90 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bf822a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562a18c838c0 con 0x562a18d222d0 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7be7400640 1 -- 172.21.15.37:0/1055971398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7bdc005020 con 0x7f7bf0103850 2024-04-26T02:00:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bef200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d21d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bef200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d21d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45168/0 (socket says 172.21.15.37:45168) 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bee800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d2c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.476+0000 7f7bef200640 1 -- 172.21.15.37:0/1441650187 learned_addr learned my addr 172.21.15.37:0/1441650187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bef200640 1 -- 172.21.15.37:0/1441650187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 msgr2=0x562a18d2c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bef200640 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d2c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bede00640 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bef200640 1 -- 172.21.15.37:0/1441650187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 msgr2=0x562a18d2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bef200640 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bef200640 1 -- 172.21.15.37:0/1441650187 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1899a210 con 0x562a18d219b0 2024-04-26T02:00:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bede00640 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:00:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bef200640 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d21d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009be0 tx=0x7f7be8030410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133adf462f45b19c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.477+0000 7f7bdb400640 1 -- 172.21.15.37:0/1441650187 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8041030 con 0x562a18d219b0 2024-04-26T02:00:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7bdb400640 1 -- 172.21.15.37:0/1441650187 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8003e20 con 0x562a18d219b0 2024-04-26T02:00:51.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7bdb400640 1 -- 172.21.15.37:0/1441650187 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8036040 con 0x562a18d219b0 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7be7400640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7bd40a3d80 0x7f7bd40a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7be7400640 1 -- 172.21.15.37:0/1055971398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7bdc0d0bb0 con 0x7f7bf0103850 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7bee800640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7bd40a3d80 0x7f7bd40a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7bf822a740 1 -- 172.21.15.37:0/1441650187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 msgr2=0x562a18d21d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.478+0000 7f7bf822a740 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d21d90 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009be0 tx=0x7f7be8030410 comp rx=0 tx=0).stop 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 -- 172.21.15.37:0/1441650187 shutdown_connections 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d2c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 --2- 172.21.15.37:0/1441650187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d21d90 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 -- 172.21.15.37:0/1441650187 >> 172.21.15.37:0/1441650187 conn(0x562a18d1dec0 msgr2=0x562a18d1e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 -- 172.21.15.37:0/1441650187 shutdown_connections 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 -- 172.21.15.37:0/1441650187 wait complete. 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bee800640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7bd40a3d80 0x7f7bd40a6240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562a18d22f50 tx=0x7f7be0009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:51.483 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.479+0000 7f7bf822a740 1 -- start start 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 Processor -- start 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d29ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d29ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a18d2f760 con 0x562a18d219b0 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x562a18c41310 con 0x562a18d2cb90 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x562a18d2f8e0 con 0x562a18d222d0 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bf822a740 10 client.0 _ll_register_callbacks cb 0x562a18c0fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.480+0000 7f7bee800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d29ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d29ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45176/0 (socket says 172.21.15.37:45176) 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 -- 172.21.15.37:0/3761162246 learned_addr learned my addr 172.21.15.37:0/3761162246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bede00640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d29ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bef200640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 -- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 msgr2=0x562a18d29ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d29ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 -- 172.21.15.37:0/3761162246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 msgr2=0x562a18d2a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 -- 172.21.15.37:0/3761162246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a18a66620 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bee800640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d29ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004620 tx=0x7f7be0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f574b1e92ac9b7e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.481+0000 7f7bcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bcfe00640 1 -- 172.21.15.37:0/3761162246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be001f640 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bcfe00640 1 -- 172.21.15.37:0/3761162246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7be001f7e0 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8003e20 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bcfe00640 1 -- 172.21.15.37:0/3761162246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7be001f9e0 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bf822a740 10 client.15270 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a18d3b960 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.482+0000 7f7bcfe00640 1 -- 172.21.15.37:0/3761162246 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8003e20 con 0x562a18d219b0 2024-04-26T02:00:51.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bcfe00640 1 -- 172.21.15.37:0/3761162246 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7be0027020 con 0x562a18d219b0 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bf822a740 10 client.15270 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bf822a740 10 client.15270 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bf822a740 20 client.15270 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bf822a740 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18d3ce70 0x562a18d3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bf822a740 4 client.15270 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a18c9b8b0 con 0x562a18d3ce70 2024-04-26T02:00:51.486 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.483+0000 7f7bede00640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18d3ce70 0x562a18d3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.484+0000 7f7bede00640 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18d3ce70 0x562a18d3f2a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=875d6e32f0684a23 server_cookie=8e58ed38271a97be in_seq=0 out_seq=0 2024-04-26T02:00:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.484+0000 7f7bcfe00640 10 client.15270 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.484+0000 7f7bcfe00640 1 -- 172.21.15.37:0/3761162246 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a18c9b8b0 con 0x562a18d3ce70 2024-04-26T02:00:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.484+0000 7f7bcfe00640 10 client.15270 handle_command_reply: tid=0 2024-04-26T02:00:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 client.15270 shutdown 2024-04-26T02:00:51.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 msgr2=0x562a18d29ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d29ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7be0004620 tx=0x7f7be0004730 comp rx=0 tx=0).stop 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 shutdown_connections 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18d3ce70 0x562a18d3f2a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d2cb90 0x562a18d2a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d29ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 --2- 172.21.15.37:0/3761162246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d29ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 >> 172.21.15.37:0/3761162246 conn(0x562a18d1dec0 msgr2=0x562a18d23bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bcfe00640 0 client.15270 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 shutdown_connections 2024-04-26T02:00:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 1 -- 172.21.15.37:0/3761162246 wait complete. 2024-04-26T02:00:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.485+0000 7f7bf822a740 20 client.15270 trim_cache size 0 max 16384 2024-04-26T02:00:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.501+0000 7f7bf822a740 1 Processor -- start 2024-04-26T02:00:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 -- start start 2024-04-26T02:00:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d219b0 0x562a18d38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d24370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 0x562a18d2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ada30 con 0x562a18d2cb90 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01b02b0 con 0x562a18d219b0 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bf822a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ae0a0 con 0x562a18d222d0 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bef200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 0x562a18d2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bef200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 0x562a18d2a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45190/0 (socket says 172.21.15.37:45190) 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bef200640 1 -- 172.21.15.37:0/2724858363 learned_addr learned my addr 172.21.15.37:0/2724858363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.502+0000 7f7bede00640 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d24370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bee800640 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d219b0 0x562a18d38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bef200640 1 -- 172.21.15.37:0/2724858363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 msgr2=0x562a18d24370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bef200640 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d24370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bef200640 1 -- 172.21.15.37:0/2724858363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d219b0 msgr2=0x562a18d38620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bef200640 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d219b0 0x562a18d38620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bef200640 1 -- 172.21.15.37:0/2724858363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a18d1e800 con 0x562a18d2cb90 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bef200640 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 0x562a18d2a180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7be8034040 tx=0x7f7be802a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8361f7d01c23646f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bdaa00640 1 -- 172.21.15.37:0/2724858363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be804e070 con 0x562a18d2cb90 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bdaa00640 1 -- 172.21.15.37:0/2724858363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8003e20 con 0x562a18d2cb90 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.503+0000 7f7bdaa00640 1 -- 172.21.15.37:0/2724858363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8036030 con 0x562a18d2cb90 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 -- 172.21.15.37:0/2724858363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 msgr2=0x562a18d2a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 0x562a18d2a180 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f7be8034040 tx=0x7f7be802a750 comp rx=0 tx=0).stop 2024-04-26T02:00:51.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 -- 172.21.15.37:0/2724858363 shutdown_connections 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d2cb90 0x562a18d2a180 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d222d0 0x562a18d24370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 --2- 172.21.15.37:0/2724858363 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d219b0 0x562a18d38620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 -- 172.21.15.37:0/2724858363 >> 172.21.15.37:0/2724858363 conn(0x562a18d1dec0 msgr2=0x562a18d29a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 -- 172.21.15.37:0/2724858363 shutdown_connections 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.504+0000 7f7bf822a740 1 -- 172.21.15.37:0/2724858363 wait complete. 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 -- start start 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 Processor -- start 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d34e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d222d0 0x562a18d32160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d2cb90 0x562a18d326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01ae770 con 0x562a18d219b0 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01afbe0 con 0x562a18d222d0 2024-04-26T02:00:51.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.505+0000 7f7bf822a740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf01aee40 con 0x562a18d2cb90 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bf822a740 10 client.0 _ll_register_callbacks cb 0x7f7be00044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d34e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d34e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45206/0 (socket says 172.21.15.37:45206) 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 -- 172.21.15.37:0/93694377 learned_addr learned my addr 172.21.15.37:0/93694377 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bede00640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d222d0 0x562a18d32160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bef200640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d2cb90 0x562a18d326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 -- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d2cb90 msgr2=0x562a18d326a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d2cb90 0x562a18d326a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 -- 172.21.15.37:0/93694377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d222d0 msgr2=0x562a18d32160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d222d0 0x562a18d32160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bee800640 1 -- 172.21.15.37:0/93694377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a18d32d90 con 0x562a18d219b0 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bef200640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d2cb90 0x562a18d326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.506+0000 7f7bede00640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d222d0 0x562a18d32160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:00:51.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bee800640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d34e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7be0019040 tx=0x7f7be0009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80934a2489ad661 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bcfe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bcfe00640 1 -- 172.21.15.37:0/93694377 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be0036030 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bcfe00640 1 -- 172.21.15.37:0/93694377 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f7be001fd90 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7be8003e20 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bcfe00640 1 -- 172.21.15.37:0/93694377 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f7be0003e50 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bf822a740 10 client.15276 fetch_fsmap learned FSMap version 19 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x562a18da3340 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.507+0000 7f7bcfe00640 1 -- 172.21.15.37:0/93694377 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7be8003e20 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bcfe00640 1 -- 172.21.15.37:0/93694377 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f7be0027040 con 0x562a18d219b0 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bf822a740 10 client.15276 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bf822a740 10 client.15276 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bf822a740 20 client.15276 populate_metadata read hostname 'smithi037' 2024-04-26T02:00:51.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bf822a740 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18da3b10 0x562a18da5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:00:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bf822a740 4 client.15276 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:00:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a18c95320 con 0x562a18da3b10 2024-04-26T02:00:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.508+0000 7f7bede00640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18da3b10 0x562a18da5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:00:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.509+0000 7f7bede00640 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18da3b10 0x562a18da5f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64206790065e4e93 server_cookie=39dc86b792af9631 in_seq=0 out_seq=0 2024-04-26T02:00:51.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.509+0000 7f7bcfe00640 10 client.15276 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.516+0000 7f7bcfe00640 1 -- 172.21.15.37:0/93694377 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a18c95320 con 0x562a18da3b10 2024-04-26T02:00:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.516+0000 7f7bcfe00640 10 client.15276 handle_command_reply: tid=0 2024-04-26T02:00:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.516+0000 7f7bf822a740 1 client.15276 shutdown 2024-04-26T02:00:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.516+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 msgr2=0x562a18d34e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.516+0000 7f7bf822a740 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d34e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7be0019040 tx=0x7f7be0009c50 comp rx=0 tx=0).stop 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 shutdown_connections 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x562a18da3b10 0x562a18da5f30 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x562a18d2cb90 0x562a18d326a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x562a18d222d0 0x562a18d32160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 --2- 172.21.15.37:0/93694377 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a18d219b0 0x562a18d34e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bcfe00640 0 client.15276 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 >> 172.21.15.37:0/93694377 conn(0x562a18d1dec0 msgr2=0x562a18d1d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 shutdown_connections 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 1 -- 172.21.15.37:0/93694377 wait complete. 2024-04-26T02:00:51.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.517+0000 7f7bf822a740 20 client.15276 trim_cache size 0 max 16384 2024-04-26T02:00:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:00:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:00:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:00:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:00:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7bd40a3d80 msgr2=0x7f7bd40a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7bd40a3d80 0x7f7bd40a6240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x562a18d22f50 tx=0x7f7be0009260 comp rx=0 tx=0).stop 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 msgr2=0x7f7bf019acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 0x7f7bf019acd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc00e9f0 tx=0x7f7bdc00edf0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 shutdown_connections 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7bd40a3d80 0x7f7bd40a6240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7bf010c350 0x7f7bf019f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7bf0103850 0x7f7bf019acd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 --2- 172.21.15.37:0/1055971398 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7bf0102f30 0x7f7bf019a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 >> 172.21.15.37:0/1055971398 conn(0x7f7bf00fc730 msgr2=0x7f7bf0100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 shutdown_connections 2024-04-26T02:00:51.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:00:51.518+0000 7f7bf4a00640 1 -- 172.21.15.37:0/1055971398 wait complete. 2024-04-26T02:00:52.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:51 smithi037 ceph-mon[27906]: pgmap v90: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 125 MiB/s wr, 35 op/s 2024-04-26T02:00:52.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:51 smithi045 ceph-mon[32520]: pgmap v90: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 125 MiB/s wr, 35 op/s 2024-04-26T02:00:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:51 smithi071 ceph-mon[32578]: pgmap v90: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 125 MiB/s wr, 35 op/s 2024-04-26T02:00:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:53 smithi037 ceph-mon[27906]: pgmap v91: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 125 MiB/s wr, 38 op/s 2024-04-26T02:00:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:53 smithi045 ceph-mon[32520]: pgmap v91: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 125 MiB/s wr, 38 op/s 2024-04-26T02:00:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:53 smithi071 ceph-mon[32578]: pgmap v91: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 125 MiB/s wr, 38 op/s 2024-04-26T02:00:56.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:55 smithi037 ceph-mon[27906]: pgmap v92: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T02:00:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:55 smithi045 ceph-mon[32520]: pgmap v92: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T02:00:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:55 smithi071 ceph-mon[32578]: pgmap v92: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T02:00:58.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:57 smithi037 ceph-mon[27906]: pgmap v93: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T02:00:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:57 smithi045 ceph-mon[32520]: pgmap v93: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T02:00:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:57 smithi071 ceph-mon[32578]: pgmap v93: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-04-26T02:01:00.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:00:59 smithi037 ceph-mon[27906]: pgmap v94: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-04-26T02:01:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:00:59 smithi045 ceph-mon[32520]: pgmap v94: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-04-26T02:01:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:00:59 smithi071 ceph-mon[32578]: pgmap v94: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-04-26T02:01:01.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:02.132 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:01 smithi037 ceph-mon[27906]: pgmap v95: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:01 smithi045 ceph-mon[32520]: pgmap v95: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:01 smithi071 ceph-mon[32578]: pgmap v95: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:04.227 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:03 smithi037 ceph-mon[27906]: pgmap v96: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-26T02:01:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:03 smithi045 ceph-mon[32520]: pgmap v96: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-26T02:01:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:03 smithi071 ceph-mon[32578]: pgmap v96: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 -- 172.21.15.37:0/3511744868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610108970 msgr2=0x7f2610108d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 --2- 172.21.15.37:0/3511744868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610108970 0x7f2610108d50 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f2604009990 tx=0x7f2604009ca0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 -- 172.21.15.37:0/3511744868 shutdown_connections 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 --2- 172.21.15.37:0/3511744868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610108970 0x7f2610108d50 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 --2- 172.21.15.37:0/3511744868 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610103330 0x7f261010f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 --2- 172.21.15.37:0/3511744868 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2610102970 0x7f2610102df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 -- 172.21.15.37:0/3511744868 >> 172.21.15.37:0/3511744868 conn(0x7f26100fe4c0 msgr2=0x7f26101008e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 -- 172.21.15.37:0/3511744868 shutdown_connections 2024-04-26T02:01:04.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.488+0000 7f2616800700 1 -- 172.21.15.37:0/3511744868 wait complete. 2024-04-26T02:01:04.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.490+0000 7f2616800700 1 Processor -- start 2024-04-26T02:01:04.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.490+0000 7f2616800700 1 -- start start 2024-04-26T02:01:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2616800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 0x7f261019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2616800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610103330 0x7f261019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2616800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2610108970 0x7f261019fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2616800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2610112bb0 con 0x7f2610103330 2024-04-26T02:01:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2616800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f2610112a30 con 0x7f2610102970 2024-04-26T02:01:04.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2616800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f2610112d30 con 0x7f2610108970 2024-04-26T02:01:04.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2615400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 0x7f261019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2615e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2610108970 0x7f261019fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2615400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 0x7f261019b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:54626/0 (socket says 172.21.15.37:54626) 2024-04-26T02:01:04.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2615400700 1 -- 172.21.15.37:0/253004936 learned_addr learned my addr 172.21.15.37:0/253004936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:04.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.491+0000 7f2614a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610103330 0x7f261019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2615400700 1 -- 172.21.15.37:0/253004936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2610108970 msgr2=0x7f261019fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2615400700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2610108970 0x7f261019fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2615400700 1 -- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610103330 msgr2=0x7f261019b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2615400700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610103330 0x7f261019b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2615400700 1 -- 172.21.15.37:0/253004936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26101a0290 con 0x7f2610102970 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2615400700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 0x7f261019b280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2604000c00 tx=0x7f26040057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604027070 con 0x7f2610102970 2024-04-26T02:01:04.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2604022470 con 0x7f2610102970 2024-04-26T02:01:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26101a0520 con 0x7f2610102970 2024-04-26T02:01:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.492+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2604017650 con 0x7f2610102970 2024-04-26T02:01:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.493+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26101a7e00 con 0x7f2610102970 2024-04-26T02:01:04.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.494+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2610069730 con 0x7f2610102970 2024-04-26T02:01:04.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.496+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f260402a070 con 0x7f2610102970 2024-04-26T02:01:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.498+0000 7f260e000700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f25f80a3e30 0x7f25f80a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.499+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f26040d14f0 con 0x7f2610102970 2024-04-26T02:01:04.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.499+0000 7f2614a00700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f25f80a3e30 0x7f25f80a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.500+0000 7f2614a00700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f25f80a3e30 0x7f25f80a62f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2610076cd0 tx=0x7f25fc009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.500+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2604013070 con 0x7f2610102970 2024-04-26T02:01:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.655+0000 7f70cfe00640 1 -- 172.21.15.37:0/2638242762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102980 msgr2=0x7f70d01069c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.655+0000 7f70cfe00640 1 --2- 172.21.15.37:0/2638242762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102980 0x7f70d01069c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009900 tx=0x7f70bc02a410 comp rx=0 tx=0).stop 2024-04-26T02:01:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 -- 172.21.15.37:0/2638242762 shutdown_connections 2024-04-26T02:01:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 --2- 172.21.15.37:0/2638242762 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d01070a0 0x7f70d010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 --2- 172.21.15.37:0/2638242762 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d0102980 0x7f70d01069c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 --2- 172.21.15.37:0/2638242762 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d0101fd0 0x7f70d01023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 -- 172.21.15.37:0/2638242762 >> 172.21.15.37:0/2638242762 conn(0x7f70d00fd890 msgr2=0x7f70d00ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 -- 172.21.15.37:0/2638242762 shutdown_connections 2024-04-26T02:01:04.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.656+0000 7f70cfe00640 1 -- 172.21.15.37:0/2638242762 wait complete. 2024-04-26T02:01:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 Processor -- start 2024-04-26T02:01:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 -- start start 2024-04-26T02:01:04.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 0x7f70d01a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d01070a0 0x7f70d01a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d019a630 0x7f70d019aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0112010 con 0x7f70d0101fd0 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0111e90 con 0x7f70d019a630 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.657+0000 7f70cfe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0112190 con 0x7f70d01070a0 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 0x7f70d01a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 0x7f70d01a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33902/0 (socket says 172.21.15.37:33902) 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70ce000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d01070a0 0x7f70d01a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70cea00640 1 -- 172.21.15.37:0/3599381093 learned_addr learned my addr 172.21.15.37:0/3599381093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70cf400640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d019a630 0x7f70d019aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70ce000640 1 -- 172.21.15.37:0/3599381093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d019a630 msgr2=0x7f70d019aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70ce000640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d019a630 0x7f70d019aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70ce000640 1 -- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 msgr2=0x7f70d01a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70ce000640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 0x7f70d01a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70ce000640 1 -- 172.21.15.37:0/3599381093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d019b230 con 0x7f70d01070a0 2024-04-26T02:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.658+0000 7f70cea00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 0x7f70d01a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:04.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.659+0000 7f70ce000640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d01070a0 0x7f70d01a0ab0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70bc02a930 tx=0x7f70bc003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.659+0000 7f70c7400640 1 -- 172.21.15.37:0/3599381093 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc03f070 con 0x7f70d01070a0 2024-04-26T02:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.659+0000 7f70c7400640 1 -- 172.21.15.37:0/3599381093 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70bc031b60 con 0x7f70d01070a0 2024-04-26T02:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.659+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d019b4c0 con 0x7f70d01070a0 2024-04-26T02:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.659+0000 7f70c7400640 1 -- 172.21.15.37:0/3599381093 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc035030 con 0x7f70d01070a0 2024-04-26T02:01:04.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.659+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d01a7600 con 0x7f70d01070a0 2024-04-26T02:01:04.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.662+0000 7f70d837e740 1 Processor -- start 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.662+0000 7f70c7400640 1 -- 172.21.15.37:0/3599381093 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f70bc042070 con 0x7f70d01070a0 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 -- start start 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d2669dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669e2d0 0x561d266a8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d26603900 con 0x561d2669d9b0 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d265a9ef0 con 0x561d2669e2d0 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.663+0000 7f70d837e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d265ff8c0 con 0x561d266a8b90 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d2669dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669e2d0 0x561d266a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d2669dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33908/0 (socket says 172.21.15.37:33908) 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 -- 172.21.15.37:0/2794022157 learned_addr learned my addr 172.21.15.37:0/2794022157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70ce000640 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 -- 172.21.15.37:0/2794022157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 msgr2=0x561d266aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 -- 172.21.15.37:0/2794022157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669e2d0 msgr2=0x561d266a8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669e2d0 0x561d266a8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cf400640 1 -- 172.21.15.37:0/2794022157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d26316210 con 0x561d2669d9b0 2024-04-26T02:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70cea00640 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669e2d0 0x561d266a8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:04.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.664+0000 7f70ce000640 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.665+0000 7f70cf400640 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d2669dd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009f50 tx=0x7f70c80118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b86c2bc4ab8ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.665+0000 7f70c7400640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f70b40a3d00 0x7f70b40a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.665+0000 7f70c7400640 1 -- 172.21.15.37:0/3599381093 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f70bc0f0d10 con 0x7f70d01070a0 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.665+0000 7f70bb400640 1 -- 172.21.15.37:0/2794022157 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8004b20 con 0x561d2669d9b0 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.665+0000 7f70bb400640 1 -- 172.21.15.37:0/2794022157 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c8004cc0 con 0x561d2669d9b0 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.665+0000 7f70cea00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f70b40a3d00 0x7f70b40a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70bb400640 1 -- 172.21.15.37:0/2794022157 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c8005020 con 0x561d2669d9b0 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 -- 172.21.15.37:0/2794022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 msgr2=0x561d2669dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d2669dd90 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f70c8009f50 tx=0x7f70c80118d0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 -- 172.21.15.37:0/2794022157 shutdown_connections 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266aaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669e2d0 0x561d266a8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 --2- 172.21.15.37:0/2794022157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d2669dd90 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 -- 172.21.15.37:0/2794022157 >> 172.21.15.37:0/2794022157 conn(0x561d26699ec0 msgr2=0x561d2669a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 -- 172.21.15.37:0/2794022157 shutdown_connections 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70cea00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f70b40a3d00 0x7f70b40a61c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x561d2669ef50 tx=0x7f70c0005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.666+0000 7f70d837e740 1 -- 172.21.15.37:0/2794022157 wait complete. 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.667+0000 7f70d837e740 1 -- start start 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.667+0000 7f70d837e740 1 Processor -- start 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.667+0000 7f70d837e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70d837e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70d837e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70d837e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d266ab7c0 con 0x561d2669d9b0 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70d837e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x561d265bd310 con 0x561d266a8b90 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70d837e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x561d266ab940 con 0x561d2669e2d0 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70d837e740 10 client.0 _ll_register_callbacks cb 0x561d2658bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266a5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33920/0 (socket says 172.21.15.37:33920) 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70cea00640 1 -- 172.21.15.37:0/1248916795 learned_addr learned my addr 172.21.15.37:0/1248916795 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:04.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70ce000640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70cf400640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70ce000640 1 -- 172.21.15.37:0/1248916795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 msgr2=0x561d266a64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70ce000640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70ce000640 1 -- 172.21.15.37:0/1248916795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 msgr2=0x561d266a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.668+0000 7f70ce000640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70ce000640 1 -- 172.21.15.37:0/1248916795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d263e2620 con 0x561d2669e2d0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70cea00640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70ce000640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a5fa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0ee430 tx=0x7f70bc02a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b6a07f9aebca0a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70afe00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70afe00640 1 -- 172.21.15.37:0/1248916795 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc049070 con 0x561d2669e2d0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70afe00640 1 -- 172.21.15.37:0/1248916795 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f70bc0b93a0 con 0x561d2669e2d0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70c8004cc0 con 0x561d2669e2d0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70afe00640 1 -- 172.21.15.37:0/1248916795 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f70bc0bcd30 con 0x561d2669e2d0 2024-04-26T02:01:04.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70d837e740 10 client.25444 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.669+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561d266ab810 con 0x561d2669e2d0 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70afe00640 1 -- 172.21.15.37:0/1248916795 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70c8004cc0 con 0x561d2669e2d0 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70afe00640 1 -- 172.21.15.37:0/1248916795 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f70bc0f5040 con 0x561d2669e2d0 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70d837e740 10 client.25444 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70d837e740 10 client.25444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70d837e740 20 client.25444 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70d837e740 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d266b8e20 0x561d266bb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70d837e740 4 client.25444 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d266178b0 con 0x561d266b8e20 2024-04-26T02:01:04.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.670+0000 7f70cea00640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d266b8e20 0x561d266bb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.671+0000 7f70cea00640 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d266b8e20 0x561d266bb2a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec85d6425b57bb79 server_cookie=a00e14d578bdb5ef in_seq=0 out_seq=0 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.671+0000 7f70afe00640 10 client.25444 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.672+0000 7f70afe00640 1 -- 172.21.15.37:0/1248916795 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561d266178b0 con 0x561d266b8e20 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.672+0000 7f70afe00640 10 client.25444 handle_command_reply: tid=0 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.672+0000 7f70d837e740 1 client.25444 shutdown 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 msgr2=0x561d266a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a5fa0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0ee430 tx=0x7f70bc02a410 comp rx=0 tx=0).stop 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 shutdown_connections 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d266b8e20 0x561d266bb2a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70afe00640 0 client.25444 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a5fa0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 --2- 172.21.15.37:0/1248916795 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 >> 172.21.15.37:0/1248916795 conn(0x561d26699ec0 msgr2=0x561d2669fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 shutdown_connections 2024-04-26T02:01:04.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 1 -- 172.21.15.37:0/1248916795 wait complete. 2024-04-26T02:01:04.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.673+0000 7f70d837e740 20 client.25444 trim_cache size 0 max 16384 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 Processor -- start 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 -- start start 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266b46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a0370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01adb10 con 0x561d2669d9b0 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01b0390 con 0x561d266a8b90 2024-04-26T02:01:04.707 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.707+0000 7f70d837e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01ae180 con 0x561d2669e2d0 2024-04-26T02:01:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266b46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70cf400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70ce000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70cea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266b46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33932/0 (socket says 172.21.15.37:33932) 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70cea00640 1 -- 172.21.15.37:0/2215148214 learned_addr learned my addr 172.21.15.37:0/2215148214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70ce000640 1 -- 172.21.15.37:0/2215148214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 msgr2=0x561d266a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70ce000640 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70ce000640 1 -- 172.21.15.37:0/2215148214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 msgr2=0x561d266b46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70ce000640 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266b46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.708+0000 7f70ce000640 1 -- 172.21.15.37:0/2215148214 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d266a08b0 con 0x561d2669e2d0 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.709+0000 7f70ce000640 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a0370 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0bccc0 tx=0x7f70bc005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=898eebd265ff3afe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.709+0000 7f70baa00640 1 -- 172.21.15.37:0/2215148214 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc049070 con 0x561d2669e2d0 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.709+0000 7f70baa00640 1 -- 172.21.15.37:0/2215148214 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70bc0d61f0 con 0x561d2669e2d0 2024-04-26T02:01:04.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.709+0000 7f70baa00640 1 -- 172.21.15.37:0/2215148214 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc0b93a0 con 0x561d2669e2d0 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 -- 172.21.15.37:0/2215148214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 msgr2=0x561d266a0370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a0370 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0bccc0 tx=0x7f70bc005c00 comp rx=0 tx=0).stop 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 -- 172.21.15.37:0/2215148214 shutdown_connections 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d266a8b90 0x561d266a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d2669e2d0 0x561d266a0370 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 --2- 172.21.15.37:0/2215148214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669d9b0 0x561d266b46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 -- 172.21.15.37:0/2215148214 >> 172.21.15.37:0/2215148214 conn(0x561d26699ec0 msgr2=0x561d266a7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 -- 172.21.15.37:0/2215148214 shutdown_connections 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 -- 172.21.15.37:0/2215148214 wait complete. 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.710+0000 7f70d837e740 1 -- start start 2024-04-26T02:01:04.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 Processor -- start 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669d9b0 0x561d266b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 0x561d266ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01ae850 con 0x561d2669e2d0 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01afcc0 con 0x561d2669d9b0 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01aef20 con 0x561d266a8b90 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70d837e740 10 client.0 _ll_register_callbacks cb 0x7f70bc0b90c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.711+0000 7f70ce000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 0x561d266ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70cf400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 0x561d266ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33948/0 (socket says 172.21.15.37:33948) 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 -- 172.21.15.37:0/2991808065 learned_addr learned my addr 172.21.15.37:0/2991808065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70cea00640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669d9b0 0x561d266b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 -- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 msgr2=0x561d266ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 -- 172.21.15.37:0/2991808065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669d9b0 msgr2=0x561d266b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669d9b0 0x561d266b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 -- 172.21.15.37:0/2991808065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d266af170 con 0x561d2669e2d0 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70cea00640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669d9b0 0x561d266b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70ce000640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 0x561d266ae390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0bafe0 tx=0x7f70bc0b8ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5b92e90caa3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.712+0000 7f70afe00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70afe00640 1 -- 172.21.15.37:0/2991808065 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc049070 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70afe00640 1 -- 172.21.15.37:0/2991808065 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f70bc0c5660 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc0d61f0 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70afe00640 1 -- 172.21.15.37:0/2991808065 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f70bc0c1030 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70d837e740 10 client.15300 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x561d266af170 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.713+0000 7f70afe00640 1 -- 172.21.15.37:0/2991808065 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70bc0d61f0 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70afe00640 1 -- 172.21.15.37:0/2991808065 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f70bc0bc180 con 0x561d2669e2d0 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70d837e740 10 client.15300 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70d837e740 10 client.15300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70d837e740 20 client.15300 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70d837e740 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d2671f720 0x561d26721b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70d837e740 4 client.15300 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d26611320 con 0x561d2671f720 2024-04-26T02:01:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.714+0000 7f70cea00640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d2671f720 0x561d26721b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.715+0000 7f70cea00640 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d2671f720 0x561d26721b10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa258c34659c2065 server_cookie=e200b3d1b03be904 in_seq=0 out_seq=0 2024-04-26T02:01:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.715+0000 7f70afe00640 10 client.15300 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70afe00640 1 -- 172.21.15.37:0/2991808065 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d26611320 con 0x561d2671f720 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70afe00640 10 client.15300 handle_command_reply: tid=0 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 client.15300 shutdown 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 msgr2=0x561d266ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 0x561d266ae390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f70bc0bafe0 tx=0x7f70bc0b8ff0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 shutdown_connections 2024-04-26T02:01:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x561d2671f720 0x561d26721b10 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x561d266a8b90 0x561d266ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d2669e2d0 0x561d266ae390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 --2- 172.21.15.37:0/2991808065 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x561d2669d9b0 0x561d266b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.721+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 >> 172.21.15.37:0/2991808065 conn(0x561d26699ec0 msgr2=0x561d266a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70afe00640 0 client.15300 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 shutdown_connections 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70d837e740 1 -- 172.21.15.37:0/2991808065 wait complete. 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70d837e740 20 client.15300 trim_cache size 0 max 16384 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:01:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:01:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:01:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f70b40a3d00 msgr2=0x7f70b40a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70cfe00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f70b40a3d00 0x7f70b40a61c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x561d2669ef50 tx=0x7f70c0005f90 comp rx=0 tx=0).stop 2024-04-26T02:01:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d01070a0 msgr2=0x7f70d01a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.722+0000 7f70cfe00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d01070a0 0x7f70d01a0ab0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f70bc02a930 tx=0x7f70bc003c10 comp rx=0 tx=0).stop 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 shutdown_connections 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f70b40a3d00 0x7f70b40a61c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f70d019a630 0x7f70d019aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f70d01070a0 0x7f70d01a0ab0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 --2- 172.21.15.37:0/3599381093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0101fd0 0x7f70d01a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 >> 172.21.15.37:0/3599381093 conn(0x7f70d00fd890 msgr2=0x7f70d00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 shutdown_connections 2024-04-26T02:01:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:04.723+0000 7f70cfe00640 1 -- 172.21.15.37:0/3599381093 wait complete. 2024-04-26T02:01:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.792+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26101a7fd0 con 0x7f2610102970 2024-04-26T02:01:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.793+0000 7f260e000700 1 -- 172.21.15.37:0/253004936 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f260409dbf0 con 0x7f2610102970 2024-04-26T02:01:04.793 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:01:04.793 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:01:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f25f80a3e30 msgr2=0x7f25f80a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f25f80a3e30 0x7f25f80a62f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2610076cd0 tx=0x7f25fc009260 comp rx=0 tx=0).stop 2024-04-26T02:01:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 msgr2=0x7f261019b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 0x7f261019b280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2604000c00 tx=0x7f26040057c0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 shutdown_connections 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f25f80a3e30 0x7f25f80a62f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f2610108970 0x7f261019fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f2610102970 0x7f261019b280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 --2- 172.21.15.37:0/253004936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2610103330 0x7f261019b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 >> 172.21.15.37:0/253004936 conn(0x7f26100fe4c0 msgr2=0x7f261010e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 shutdown_connections 2024-04-26T02:01:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:04.796+0000 7f2616800700 1 -- 172.21.15.37:0/253004936 wait complete. 2024-04-26T02:01:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:01:05.198 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:04 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/253004936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:05.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:04 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/253004936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:05.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:04 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/253004936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:05.751 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db --set-id cf94f552 --release --await' 2024-04-26T02:01:05.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db --set-id cf94f552 --release --await 2024-04-26T02:01:06.204 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:05 smithi045 ceph-mon[32520]: pgmap v97: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:01:06.357 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:05 smithi037 ceph-mon[27906]: pgmap v97: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:01:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:05 smithi071 ceph-mon[32578]: pgmap v97: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:01:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:07 smithi045 ceph-mon[32520]: pgmap v98: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:01:08.256 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:07 smithi037 ceph-mon[27906]: pgmap v98: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:01:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:07 smithi071 ceph-mon[32578]: pgmap v98: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:01:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 -- 172.21.15.37:0/1517964453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868109fa0 msgr2=0x7ff868111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 --2- 172.21.15.37:0/1517964453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868109fa0 0x7ff868111b70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b0d0 tx=0x7ff86000b3e0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 -- 172.21.15.37:0/1517964453 shutdown_connections 2024-04-26T02:01:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 --2- 172.21.15.37:0/1517964453 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868109fa0 0x7ff868111b70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 --2- 172.21.15.37:0/1517964453 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868104ec0 0x7ff8681052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 --2- 172.21.15.37:0/1517964453 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868105870 0x7ff8681098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.651+0000 7ff86d800700 1 -- 172.21.15.37:0/1517964453 >> 172.21.15.37:0/1517964453 conn(0x7ff868100750 msgr2=0x7ff868102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.652+0000 7ff86d800700 1 -- 172.21.15.37:0/1517964453 shutdown_connections 2024-04-26T02:01:08.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.652+0000 7ff86d800700 1 -- 172.21.15.37:0/1517964453 wait complete. 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 Processor -- start 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 -- start start 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 0x7ff86819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868105870 0x7ff86819b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868109fa0 0x7ff86819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff868114e90 con 0x7ff868105870 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff868114d10 con 0x7ff868109fa0 2024-04-26T02:01:08.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.653+0000 7ff86d800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff868115010 con 0x7ff868104ec0 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 0x7ff86819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868105870 0x7ff86819b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 0x7ff86819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57700/0 (socket says 172.21.15.37:57700) 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 -- 172.21.15.37:0/693328857 learned_addr learned my addr 172.21.15.37:0/693328857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff867400700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868109fa0 0x7ff86819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 -- 172.21.15.37:0/693328857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868109fa0 msgr2=0x7ff86819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868109fa0 0x7ff86819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 -- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868105870 msgr2=0x7ff86819b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868105870 0x7ff86819b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff866a00700 1 -- 172.21.15.37:0/693328857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8681a02b0 con 0x7ff868104ec0 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff867400700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868109fa0 0x7ff86819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.654+0000 7ff85fe00700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868105870 0x7ff86819b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.655+0000 7ff866a00700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 0x7ff86819b2a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff85800e9a0 tx=0x7ff85800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.655+0000 7ff85f400700 1 -- 172.21.15.37:0/693328857 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85800ca10 con 0x7ff868104ec0 2024-04-26T02:01:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.655+0000 7ff85f400700 1 -- 172.21.15.37:0/693328857 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85800cbb0 con 0x7ff868104ec0 2024-04-26T02:01:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.655+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8681a05a0 con 0x7ff868104ec0 2024-04-26T02:01:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.655+0000 7ff85f400700 1 -- 172.21.15.37:0/693328857 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858018680 con 0x7ff868104ec0 2024-04-26T02:01:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.655+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8681a7e20 con 0x7ff868104ec0 2024-04-26T02:01:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.658+0000 7ff85f400700 1 -- 172.21.15.37:0/693328857 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff858010430 con 0x7ff868104ec0 2024-04-26T02:01:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.660+0000 7ff85f400700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff84c0a3e30 0x7ff84c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.660+0000 7ff8780e4b80 1 Processor -- start 2024-04-26T02:01:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.660+0000 7ff85f400700 1 -- 172.21.15.37:0/693328857 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff8580d0970 con 0x7ff868104ec0 2024-04-26T02:01:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.660+0000 7ff85fe00700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff84c0a3e30 0x7ff84c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.660+0000 7ff8780e4b80 1 -- start start 2024-04-26T02:01:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f409120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f413f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f4168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563d4f35bf90 con 0x563d4f4144d0 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d4f302830 con 0x563d4f408d40 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563d4f357f50 con 0x563d4f409660 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff85fe00700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff84c0a3e30 0x7ff84c0a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff850009680 tx=0x7ff850009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff867400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f4168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f413f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff866a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f409120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff867400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f4168c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33980/0 (socket says 172.21.15.37:33980) 2024-04-26T02:01:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff867400700 1 -- 172.21.15.37:0/3078112829 learned_addr learned my addr 172.21.15.37:0/3078112829 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff866a00700 1 -- 172.21.15.37:0/3078112829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 msgr2=0x563d4f413f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.661+0000 7ff866a00700 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f413f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff866a00700 1 -- 172.21.15.37:0/3078112829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 msgr2=0x563d4f4168c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff866a00700 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f4168c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff866a00700 1 -- 172.21.15.37:0/3078112829 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4f2f8780 con 0x563d4f408d40 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff866a00700 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f409120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ce090 tx=0x7ff85809c550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac9172184f720e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff857e00700 1 -- 172.21.15.37:0/3078112829 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a5550 con 0x563d4f408d40 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff857e00700 1 -- 172.21.15.37:0/3078112829 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85809abe0 con 0x563d4f408d40 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.662+0000 7ff857e00700 1 -- 172.21.15.37:0/3078112829 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a52e0 con 0x563d4f408d40 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 -- 172.21.15.37:0/3078112829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 msgr2=0x563d4f409120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f409120 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ce090 tx=0x7ff85809c550 comp rx=0 tx=0).stop 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 -- 172.21.15.37:0/3078112829 shutdown_connections 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f4168c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f409120 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/3078112829 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f413f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 -- 172.21.15.37:0/3078112829 >> 172.21.15.37:0/3078112829 conn(0x563d4f338050 msgr2=0x563d4f405250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 -- 172.21.15.37:0/3078112829 shutdown_connections 2024-04-26T02:01:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.663+0000 7ff8780e4b80 1 -- 172.21.15.37:0/3078112829 wait complete. 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.664+0000 7ff8780e4b80 1 -- start start 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.664+0000 7ff8780e4b80 1 Processor -- start 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f413cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f409660 0x563d4f410fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f4144d0 0x563d4f4114f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563d4f416ea0 con 0x563d4f409660 2024-04-26T02:01:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x563d4f315960 con 0x563d4f408d40 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x563d4f417020 con 0x563d4f4144d0 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff8780e4b80 10 client.0 _ll_register_callbacks cb 0x563d4f2f4820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f409660 0x563d4f410fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff866a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f413cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f409660 0x563d4f410fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33996/0 (socket says 172.21.15.37:33996) 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff85fe00700 1 -- 172.21.15.37:0/1612728281 learned_addr learned my addr 172.21.15.37:0/1612728281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.665+0000 7ff867400700 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f4144d0 0x563d4f4114f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff866a00700 1 -- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f4144d0 msgr2=0x563d4f4114f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff866a00700 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f4144d0 0x563d4f4114f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff866a00700 1 -- 172.21.15.37:0/1612728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f409660 msgr2=0x563d4f410fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff866a00700 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f409660 0x563d4f410fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff866a00700 1 -- 172.21.15.37:0/1612728281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4f2a99d0 con 0x563d4f408d40 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff866a00700 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f413cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ce0c0 tx=0x7ff8580a0d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa118ddca4ad92e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff854c00700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:01:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff854c00700 1 -- 172.21.15.37:0/1612728281 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a50c0 con 0x563d4f408d40 2024-04-26T02:01:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff854c00700 1 -- 172.21.15.37:0/1612728281 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff8580a5260 con 0x563d4f408d40 2024-04-26T02:01:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.666+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85809abe0 con 0x563d4f408d40 2024-04-26T02:01:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff854c00700 1 -- 172.21.15.37:0/1612728281 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff85809bd30 con 0x563d4f408d40 2024-04-26T02:01:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 10 client.25403 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563d4f2f2bf0 con 0x563d4f408d40 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff854c00700 1 -- 172.21.15.37:0/1612728281 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85809abe0 con 0x563d4f408d40 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff854c00700 1 -- 172.21.15.37:0/1612728281 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ff85809b7a0 con 0x563d4f408d40 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 10 client.25403 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 10 client.25403 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 20 client.25403 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f423b10 0x563d4f425f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 4 client.25403 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.667+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d4f369780 con 0x563d4f423b10 2024-04-26T02:01:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.668+0000 7ff85fe00700 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f423b10 0x563d4f425f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.668+0000 7ff85fe00700 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f423b10 0x563d4f425f00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6df77ad26d64f164 server_cookie=64e2ba4faee23935 in_seq=0 out_seq=0 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.668+0000 7ff854c00700 10 client.25403 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.669+0000 7ff854c00700 1 -- 172.21.15.37:0/1612728281 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d4f369780 con 0x563d4f423b10 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.669+0000 7ff854c00700 10 client.25403 handle_command_reply: tid=0 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.669+0000 7ff8780e4b80 1 client.25403 shutdown 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 msgr2=0x563d4f413cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f413cb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff8580ce0c0 tx=0x7ff8580a0d50 comp rx=0 tx=0).stop 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 shutdown_connections 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f423b10 0x563d4f425f00 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f4144d0 0x563d4f4114f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f413cb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1612728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f409660 0x563d4f410fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff854c00700 0 client.25403 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:08.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 >> 172.21.15.37:0/1612728281 conn(0x563d4f338050 msgr2=0x563d4f404e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:08.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 shutdown_connections 2024-04-26T02:01:08.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1612728281 wait complete. 2024-04-26T02:01:08.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.670+0000 7ff8780e4b80 20 client.25403 trim_cache size 0 max 16384 2024-04-26T02:01:08.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 Processor -- start 2024-04-26T02:01:08.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 -- start start 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f405410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f412e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f428f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ad440 con 0x563d4f4144d0 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681afcc0 con 0x563d4f408d40 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.692+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681adab0 con 0x563d4f409660 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff866a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f405410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f412e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f412e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:57740/0 (socket says 172.21.15.37:57740) 2024-04-26T02:01:08.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff85fe00700 1 -- 172.21.15.37:0/2386350662 learned_addr learned my addr 172.21.15.37:0/2386350662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:08.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff867400700 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f428f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff866a00700 1 -- 172.21.15.37:0/2386350662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 msgr2=0x563d4f412e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff866a00700 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f412e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff866a00700 1 -- 172.21.15.37:0/2386350662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 msgr2=0x563d4f428f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff866a00700 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f428f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff866a00700 1 -- 172.21.15.37:0/2386350662 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d4f409bf0 con 0x563d4f408d40 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.693+0000 7ff85fe00700 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f412e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.694+0000 7ff866a00700 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f405410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8580992b0 tx=0x7ff85809c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5707f6ecbbaa0761 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:08.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.694+0000 7ff857400700 1 -- 172.21.15.37:0/2386350662 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85809abe0 con 0x563d4f408d40 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.694+0000 7ff857400700 1 -- 172.21.15.37:0/2386350662 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85809bed0 con 0x563d4f408d40 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.694+0000 7ff857400700 1 -- 172.21.15.37:0/2386350662 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8580a5320 con 0x563d4f408d40 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.694+0000 7ff8780e4b80 1 -- 172.21.15.37:0/2386350662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 msgr2=0x563d4f405410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.694+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f405410 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff8580992b0 tx=0x7ff85809c960 comp rx=0 tx=0).stop 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 -- 172.21.15.37:0/2386350662 shutdown_connections 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f4144d0 0x563d4f428f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f408d40 0x563d4f405410 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/2386350662 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f412e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 -- 172.21.15.37:0/2386350662 >> 172.21.15.37:0/2386350662 conn(0x563d4f338050 msgr2=0x563d4f41fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 -- 172.21.15.37:0/2386350662 shutdown_connections 2024-04-26T02:01:08.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 -- 172.21.15.37:0/2386350662 wait complete. 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 -- start start 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.695+0000 7ff8780e4b80 1 Processor -- start 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 0x563d4f483840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f483d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f4144d0 0x563d4f47dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ae180 con 0x563d4f408d40 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681af5f0 con 0x563d4f4144d0 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681ae850 con 0x563d4f409660 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff8780e4b80 10 client.0 _ll_register_callbacks cb 0x563d4f2f4820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:08.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff866a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 0x563d4f483840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff85fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f483d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff866a00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 0x563d4f483840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34006/0 (socket says 172.21.15.37:34006) 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff866a00700 1 -- 172.21.15.37:0/1114935493 learned_addr learned my addr 172.21.15.37:0/1114935493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.696+0000 7ff867400700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f4144d0 0x563d4f47dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff866a00700 1 -- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 msgr2=0x563d4f483d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff866a00700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f483d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff866a00700 1 -- 172.21.15.37:0/1114935493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f4144d0 msgr2=0x563d4f47dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff866a00700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f4144d0 0x563d4f47dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff866a00700 1 -- 172.21.15.37:0/1114935493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d4f405d20 con 0x563d4f408d40 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff85fe00700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f483d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff866a00700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 0x563d4f483840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff8580a5710 tx=0x7ff8580a0d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4728d7daf844c5e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff854c00700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff854c00700 1 -- 172.21.15.37:0/1114935493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85807f030 con 0x563d4f408d40 2024-04-26T02:01:08.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff854c00700 1 -- 172.21.15.37:0/1114935493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ff8580a50c0 con 0x563d4f408d40 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.697+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff85809bed0 con 0x563d4f408d40 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff854c00700 1 -- 172.21.15.37:0/1114935493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7ff8580a52c0 con 0x563d4f408d40 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 10 client.15318 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x563d4f405d20 con 0x563d4f408d40 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff854c00700 1 -- 172.21.15.37:0/1114935493 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85809bed0 con 0x563d4f408d40 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff854c00700 1 -- 172.21.15.37:0/1114935493 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7ff8580a2020 con 0x563d4f408d40 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 10 client.15318 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 10 client.15318 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 20 client.15318 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f48b110 0x563d4f48d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 4 client.15318 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cf94f552", "release": true, "await": true}] 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.698+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cf94f552", "release": true, "await": true}) -- 0x563d4f363820 con 0x563d4f48b110 2024-04-26T02:01:08.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.699+0000 7ff85fe00700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f48b110 0x563d4f48d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.699+0000 7ff85fe00700 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f48b110 0x563d4f48d500 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=283473b2f65eb748 server_cookie=1637d7a17f7c51a0 in_seq=0 out_seq=0 2024-04-26T02:01:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.699+0000 7ff854c00700 10 client.15318 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff854c00700 1 -- 172.21.15.37:0/1114935493 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x563d4f363820 con 0x563d4f48b110 2024-04-26T02:01:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff854c00700 10 client.15318 handle_command_reply: tid=0 2024-04-26T02:01:08.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 client.15318 shutdown 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 msgr2=0x563d4f483840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 0x563d4f483840 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff8580a5710 tx=0x7ff8580a0d50 comp rx=0 tx=0).stop 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 shutdown_connections 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x563d4f48b110 0x563d4f48d500 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563d4f408d40 0x563d4f483840 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x563d4f4144d0 0x563d4f47dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 --2- 172.21.15.37:0/1114935493 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x563d4f409660 0x563d4f483d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff854c00700 0 client.15318 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 >> 172.21.15.37:0/1114935493 conn(0x563d4f338050 msgr2=0x563d4f41fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 shutdown_connections 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 1 -- 172.21.15.37:0/1114935493 wait complete. 2024-04-26T02:01:08.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.700+0000 7ff8780e4b80 20 client.15318 trim_cache size 0 max 16384 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 8, 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "cf94f552": { 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 8, 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:01:08.703 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:08.704 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff84c0a3e30 msgr2=0x7ff84c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff84c0a3e30 0x7ff84c0a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff850009680 tx=0x7ff850009260 comp rx=0 tx=0).stop 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 msgr2=0x7ff86819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 0x7ff86819b2a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff85800e9a0 tx=0x7ff85800ecb0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 shutdown_connections 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ff84c0a3e30 0x7ff84c0a62f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ff868109fa0 0x7ff86819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ff868104ec0 0x7ff86819b2a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 --2- 172.21.15.37:0/693328857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff868105870 0x7ff86819b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:08.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 >> 172.21.15.37:0/693328857 conn(0x7ff868100750 msgr2=0x7ff868101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:08.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 shutdown_connections 2024-04-26T02:01:08.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:08.701+0000 7ff86d800700 1 -- 172.21.15.37:0/693328857 wait complete. 2024-04-26T02:01:09.674 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cf94f552', seconds elapsed: 49.0 2024-04-26T02:01:09.675 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T02:01:09.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:09 smithi045 ceph-mon[32520]: pgmap v99: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-04-26T02:01:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:09 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:10.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:09 smithi037 ceph-mon[27906]: pgmap v99: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-04-26T02:01:10.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:09 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:09 smithi071 ceph-mon[32578]: pgmap v99: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 23 KiB/s wr, 18 op/s 2024-04-26T02:01:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:09 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:01:11.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:10 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:11.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:10 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:11.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:10 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:01:11.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:10 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:11.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:10 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:11.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:10 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:01:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:10 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:01:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:10 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:01:11.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:10 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:01:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:11 smithi045 ceph-mon[32520]: pgmap v100: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T02:01:12.309 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:11 smithi037 ceph-mon[27906]: pgmap v100: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T02:01:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:11 smithi071 ceph-mon[32578]: pgmap v100: 97 pgs: 97 active+clean; 9.2 GiB data, 28 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-26T02:01:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.791+0000 7fb8b8000700 1 -- 172.21.15.37:0/132137399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0102a90 msgr2=0x7fb8b0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.791+0000 7fb8afe00700 1 -- 172.21.15.37:0/132137399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c00bb00 con 0x7fb8b0102a90 2024-04-26T02:01:12.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.791+0000 7fb8b8000700 1 --2- 172.21.15.37:0/132137399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0102a90 0x7fb8b0102f10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb89c009990 tx=0x7fb89c009ca0 comp rx=0 tx=0).stop 2024-04-26T02:01:12.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.792+0000 7fb8b8000700 1 -- 172.21.15.37:0/132137399 shutdown_connections 2024-04-26T02:01:12.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.792+0000 7fb8b8000700 1 --2- 172.21.15.37:0/132137399 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 0x7fb8b010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:12.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.792+0000 7fb8b8000700 1 --2- 172.21.15.37:0/132137399 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0108a50 0x7fb8b0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:12.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.792+0000 7fb8b8000700 1 --2- 172.21.15.37:0/132137399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0102a90 0x7fb8b0102f10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:12.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.792+0000 7fb8b8000700 1 -- 172.21.15.37:0/132137399 >> 172.21.15.37:0/132137399 conn(0x7fb8b0079ba0 msgr2=0x7fb8b0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.793+0000 7fb8b8000700 1 -- 172.21.15.37:0/132137399 shutdown_connections 2024-04-26T02:01:12.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.793+0000 7fb8b8000700 1 -- 172.21.15.37:0/132137399 wait complete. 2024-04-26T02:01:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.794+0000 7fb8b8000700 1 Processor -- start 2024-04-26T02:01:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 -- start start 2024-04-26T02:01:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0102a90 0x7fb8b019bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 0x7fb8b019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0108a50 0x7fb8b010e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0112d20 con 0x7fb8b0108a50 2024-04-26T02:01:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0112ba0 con 0x7fb8b0102a90 2024-04-26T02:01:12.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b8000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b0112ea0 con 0x7fb8b0103450 2024-04-26T02:01:12.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.795+0000 7fb8b5600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0102a90 0x7fb8b019bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:12.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b5600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0102a90 0x7fb8b019bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:33900/0 (socket says 172.21.15.37:33900) 2024-04-26T02:01:12.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b5600700 1 -- 172.21.15.37:0/3202525822 learned_addr learned my addr 172.21.15.37:0/3202525822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:12.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b4c00700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 0x7fb8b019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:12.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b4c00700 1 -- 172.21.15.37:0/3202525822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0102a90 msgr2=0x7fb8b019bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b4c00700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0102a90 0x7fb8b019bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b4c00700 1 -- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0108a50 msgr2=0x7fb8b010e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b4c00700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0108a50 0x7fb8b010e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.796+0000 7fb8b4c00700 1 -- 172.21.15.37:0/3202525822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b010ef80 con 0x7fb8b0103450 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.797+0000 7fb8b4c00700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 0x7fb8b019c000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0098e0 tx=0x7fb89c00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.797+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c027070 con 0x7fb8b0103450 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.797+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb89c022920 con 0x7fb8b0103450 2024-04-26T02:01:12.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.797+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89c017630 con 0x7fb8b0103450 2024-04-26T02:01:12.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.797+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b010f210 con 0x7fb8b0103450 2024-04-26T02:01:12.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.798+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b019fa10 con 0x7fb8b0103450 2024-04-26T02:01:12.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.799+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8b0104180 con 0x7fb8b0103450 2024-04-26T02:01:12.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.804+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb89c009d10 con 0x7fb8b0103450 2024-04-26T02:01:12.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.806+0000 7fb8ae000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8980a3e70 0x7fb8980a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:12.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.806+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fb89c0d1cc0 con 0x7fb8b0103450 2024-04-26T02:01:12.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.807+0000 7fb8b5600700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8980a3e70 0x7fb8980a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:12.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.807+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb89c013070 con 0x7fb8b0103450 2024-04-26T02:01:12.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:12.808+0000 7fb8b5600700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8980a3e70 0x7fb8980a6330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40098f0 tx=0x7fb8a4008040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:13.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.282+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8b0069730 con 0x7fb8b0103450 2024-04-26T02:01:13.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.283+0000 7fb8ae000700 1 -- 172.21.15.37:0/3202525822 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fb8b0069730 con 0x7fb8b0103450 2024-04-26T02:01:13.282 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:01:13.283 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:01:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.287+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8980a3e70 msgr2=0x7fb8980a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.287+0000 7fb8b8000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8980a3e70 0x7fb8980a6330 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb8a40098f0 tx=0x7fb8a4008040 comp rx=0 tx=0).stop 2024-04-26T02:01:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.287+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 msgr2=0x7fb8b019c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:13.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.287+0000 7fb8b8000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 0x7fb8b019c000 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb89c0098e0 tx=0x7fb89c00f740 comp rx=0 tx=0).stop 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 shutdown_connections 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8b0108a50 0x7fb8b010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8980a3e70 0x7fb8980a6330 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8b0102a90 0x7fb8b019bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 --2- 172.21.15.37:0/3202525822 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb8b0103450 0x7fb8b019c000 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 >> 172.21.15.37:0/3202525822 conn(0x7fb8b0079ba0 msgr2=0x7fb8b0104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 shutdown_connections 2024-04-26T02:01:13.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:13.289+0000 7fb8b8000700 1 -- 172.21.15.37:0/3202525822 wait complete. 2024-04-26T02:01:13.292 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:01:14.000 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:14 smithi037 ceph-mon[27906]: pgmap v101: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 59 MiB/s wr, 32 op/s 2024-04-26T02:01:14.000 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:14 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3202525822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:14 smithi045 ceph-mon[32520]: pgmap v101: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 59 MiB/s wr, 32 op/s 2024-04-26T02:01:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:14 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3202525822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:14.345 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T02:01:14.345 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T02:01:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:14 smithi071 ceph-mon[32578]: pgmap v101: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 59 MiB/s wr, 32 op/s 2024-04-26T02:01:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:14 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3202525822' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:15.654 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:01:15.655 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 initial writers = 175371.77 kB/sec 2024-04-26T02:01:15.655 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 initial writers = 175369.41 kB/sec 2024-04-26T02:01:15.655 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 175371.77 kB/sec 2024-04-26T02:01:15.655 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 175371.77 kB/sec 2024-04-26T02:01:15.655 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 175371.77 kB/sec 2024-04-26T02:01:15.655 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 10485760.00 kB 2024-04-26T02:01:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:16 smithi071 ceph-mon[32578]: pgmap v102: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 59 MiB/s wr, 17 op/s 2024-04-26T02:01:16.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:16 smithi037 ceph-mon[27906]: pgmap v102: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 59 MiB/s wr, 17 op/s 2024-04-26T02:01:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:16 smithi045 ceph-mon[32520]: pgmap v102: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 937 B/s rd, 59 MiB/s wr, 17 op/s 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 -- 172.21.15.37:0/2547378479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc1031c0 msgr2=0x7f11cc1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2547378479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc1031c0 0x7f11cc1035a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f11bc009900 tx=0x7f11bc02a490 comp rx=0 tx=0).stop 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 -- 172.21.15.37:0/2547378479 shutdown_connections 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2547378479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 0x7f11cc10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2547378479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc103ae0 0x7f11cc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2547378479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc1031c0 0x7f11cc1035a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 -- 172.21.15.37:0/2547378479 >> 172.21.15.37:0/2547378479 conn(0x7f11cc077f70 msgr2=0x7f11cc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 -- 172.21.15.37:0/2547378479 shutdown_connections 2024-04-26T02:01:17.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.861+0000 7f11d2a00640 1 -- 172.21.15.37:0/2547378479 wait complete. 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.862+0000 7f11d2a00640 1 Processor -- start 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.862+0000 7f11d2a00640 1 -- start start 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d2a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 0x7f11cc1a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d2a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc103ae0 0x7f11cc1a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d2a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 0x7f11cc19a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d2a00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc112210 con 0x7f11cc103ae0 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d2a00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc112090 con 0x7f11cc1031c0 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d2a00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc112390 con 0x7f11cc10c590 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11cb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc103ae0 0x7f11cc1a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 0x7f11cc1a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 0x7f11cc1a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:33924/0 (socket says 172.21.15.37:33924) 2024-04-26T02:01:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11cbe00640 1 -- 172.21.15.37:0/2240703458 learned_addr learned my addr 172.21.15.37:0/2240703458 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11d0a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 0x7f11cc19a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.863+0000 7f11cbe00640 1 -- 172.21.15.37:0/2240703458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 msgr2=0x7f11cc19a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11cbe00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 0x7f11cc19a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11cbe00640 1 -- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc103ae0 msgr2=0x7f11cc1a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11cbe00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc103ae0 0x7f11cc1a0c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11cbe00640 1 -- 172.21.15.37:0/2240703458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc19b000 con 0x7f11cc1031c0 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11d0a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 0x7f11cc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11cbe00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 0x7f11cc1a06c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005ab0 tx=0x7f11bc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11c8c00640 1 -- 172.21.15.37:0/2240703458 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc03f070 con 0x7f11cc1031c0 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc19b290 con 0x7f11cc1031c0 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11c8c00640 1 -- 172.21.15.37:0/2240703458 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11bc031b60 con 0x7f11cc1031c0 2024-04-26T02:01:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11c8c00640 1 -- 172.21.15.37:0/2240703458 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc035030 con 0x7f11cc1031c0 2024-04-26T02:01:17.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.864+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc1a7510 con 0x7f11cc1031c0 2024-04-26T02:01:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 Processor -- start 2024-04-26T02:01:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 -- start start 2024-04-26T02:01:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928327820 0x55e928327c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928328140 0x55e9283324c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928334df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e92828d900 con 0x55e928327820 2024-04-26T02:01:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e928233ef0 con 0x55e928328140 2024-04-26T02:01:17.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11d59ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e9282898c0 con 0x55e928332a00 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928334df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928334df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35702/0 (socket says 172.21.15.37:35702) 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.868+0000 7f11cb400640 1 -- 172.21.15.37:0/1017993235 learned_addr learned my addr 172.21.15.37:0/1017993235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11d0a00640 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928327820 0x55e928327c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11cb400640 1 -- 172.21.15.37:0/1017993235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928328140 msgr2=0x55e9283324c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11cb400640 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928328140 0x55e9283324c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11cb400640 1 -- 172.21.15.37:0/1017993235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928327820 msgr2=0x55e928327c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11cb400640 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928327820 0x55e928327c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11cb400640 1 -- 172.21.15.37:0/1017993235 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e927fa0210 con 0x55e928332a00 2024-04-26T02:01:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11cb400640 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928334df0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009c60 tx=0x7f11b8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5b294b247b9cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11c8c00640 1 -- 172.21.15.37:0/2240703458 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11bc042070 con 0x7f11cc1031c0 2024-04-26T02:01:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11c1200640 1 -- 172.21.15.37:0/1017993235 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8004230 con 0x55e928332a00 2024-04-26T02:01:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11c1200640 1 -- 172.21.15.37:0/1017993235 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b80043d0 con 0x55e928332a00 2024-04-26T02:01:17.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.869+0000 7f11c1200640 1 -- 172.21.15.37:0/1017993235 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8004c70 con 0x55e928332a00 2024-04-26T02:01:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 -- 172.21.15.37:0/1017993235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 msgr2=0x55e928334df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928334df0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009c60 tx=0x7f11b8011810 comp rx=0 tx=0).stop 2024-04-26T02:01:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 -- 172.21.15.37:0/1017993235 shutdown_connections 2024-04-26T02:01:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928334df0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928328140 0x55e9283324c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1017993235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928327820 0x55e928327c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.870+0000 7f11d59ba740 1 -- 172.21.15.37:0/1017993235 >> 172.21.15.37:0/1017993235 conn(0x55e928323d30 msgr2=0x55e928324110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11d59ba740 1 -- 172.21.15.37:0/1017993235 shutdown_connections 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11d59ba740 1 -- 172.21.15.37:0/1017993235 wait complete. 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11c8c00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f11b00a3c20 0x7f11b00a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11d59ba740 1 -- start start 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11c8c00640 1 -- 172.21.15.37:0/2240703458 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f11bc0f0f90 con 0x7f11cc1031c0 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11d59ba740 1 Processor -- start 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.871+0000 7f11d0a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f11b00a3c20 0x7f11b00a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92832f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928328140 0x55e92832fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928330410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e928335630 con 0x55e928328140 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55e928247310 con 0x55e928327820 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55e9283357b0 con 0x55e928332a00 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d59ba740 10 client.0 _ll_register_callbacks cb 0x55e928215cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11cb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928328140 0x55e92832fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d0a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f11b00a3c20 0x7f11b00a60e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11c40099a0 tx=0x7f11c4009500 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11cb400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928328140 0x55e92832fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46534/0 (socket says 172.21.15.37:46534) 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11cb400640 1 -- 172.21.15.37:0/1687614022 learned_addr learned my addr 172.21.15.37:0/1687614022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92832f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.872+0000 7f11d0a00640 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928330410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11cbe00640 1 -- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 msgr2=0x55e928330410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11cbe00640 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928330410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11cbe00640 1 -- 172.21.15.37:0/1687614022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928328140 msgr2=0x55e92832fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11cbe00640 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928328140 0x55e92832fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11cbe00640 1 -- 172.21.15.37:0/1687614022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92806c620 con 0x55e928327820 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11cbe00640 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92832f990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0ee710 tx=0x7f11bc0bcb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d0365007fabc93d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11b5600640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11b5600640 1 -- 172.21.15.37:0/1687614022 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0eee60 con 0x55e928327820 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11b5600640 1 -- 172.21.15.37:0/1687614022 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f11bc0b95d0 con 0x55e928327820 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.873+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b80043d0 con 0x55e928327820 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11b5600640 1 -- 172.21.15.37:0/1687614022 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f11bc049070 con 0x55e928327820 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 10 client.25421 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e9283418d0 con 0x55e928327820 2024-04-26T02:01:17.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11b5600640 1 -- 172.21.15.37:0/1687614022 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11b80043d0 con 0x55e928327820 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11b5600640 1 -- 172.21.15.37:0/1687614022 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f11bc03b460 con 0x55e928327820 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 10 client.25421 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 10 client.25421 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 20 client.25421 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e928342d30 0x55e928345180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 4 client.25421 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.874+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9282a18b0 con 0x55e928342d30 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.875+0000 7f11cb400640 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e928342d30 0x55e928345180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.875+0000 7f11cb400640 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e928342d30 0x55e928345180 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f46c0cf671e1af9b server_cookie=bb11e878bd25eb0e in_seq=0 out_seq=0 2024-04-26T02:01:17.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.875+0000 7f11b5600640 10 client.25421 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.876+0000 7f11b5600640 1 -- 172.21.15.37:0/1687614022 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e9282a18b0 con 0x55e928342d30 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.876+0000 7f11b5600640 10 client.25421 handle_command_reply: tid=0 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.876+0000 7f11d59ba740 1 client.25421 shutdown 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.876+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 msgr2=0x55e92832f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.876+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92832f990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11bc0ee710 tx=0x7f11bc0bcb60 comp rx=0 tx=0).stop 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 shutdown_connections 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e928342d30 0x55e928345180 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928332a00 0x55e928330410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928328140 0x55e92832fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 --2- 172.21.15.37:0/1687614022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92832f990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11b5600640 0 client.25421 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 >> 172.21.15.37:0/1687614022 conn(0x55e928323d30 msgr2=0x55e928329ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 shutdown_connections 2024-04-26T02:01:17.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 1 -- 172.21.15.37:0/1687614022 wait complete. 2024-04-26T02:01:17.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.877+0000 7f11d59ba740 20 client.25421 trim_cache size 0 max 16384 2024-04-26T02:01:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 Processor -- start 2024-04-26T02:01:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 -- start start 2024-04-26T02:01:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92833e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.892 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e92832a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928322d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1adcd0 con 0x55e928332a00 2024-04-26T02:01:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1b0550 con 0x55e928327820 2024-04-26T02:01:17.893 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11d59ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1ae340 con 0x55e928328140 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92833e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11cbe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92833e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:33946/0 (socket says 172.21.15.37:33946) 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.893+0000 7f11cbe00640 1 -- 172.21.15.37:0/3221410297 learned_addr learned my addr 172.21.15.37:0/3221410297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11d0a00640 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928322d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11cb400640 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e92832a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11d0a00640 1 -- 172.21.15.37:0/3221410297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 msgr2=0x55e92832a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11d0a00640 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e92832a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11d0a00640 1 -- 172.21.15.37:0/3221410297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 msgr2=0x55e92833e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11d0a00640 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92833e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.894+0000 7f11d0a00640 1 -- 172.21.15.37:0/3221410297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e928323860 con 0x55e928332a00 2024-04-26T02:01:17.894 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.895+0000 7f11d0a00640 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928322d10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11cc06a850 tx=0x7f11c4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c94f7dffde8ac50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.895+0000 7f11b7e00640 1 -- 172.21.15.37:0/3221410297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c4004d20 con 0x55e928332a00 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.895+0000 7f11b7e00640 1 -- 172.21.15.37:0/3221410297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c401ca30 con 0x55e928332a00 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.895+0000 7f11b7e00640 1 -- 172.21.15.37:0/3221410297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c40055e0 con 0x55e928332a00 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 -- 172.21.15.37:0/3221410297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 msgr2=0x55e928322d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928322d10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f11cc06a850 tx=0x7f11c4019000 comp rx=0 tx=0).stop 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 -- 172.21.15.37:0/3221410297 shutdown_connections 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928322d10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e92832a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3221410297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e92833e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 -- 172.21.15.37:0/3221410297 >> 172.21.15.37:0/3221410297 conn(0x55e928323d30 msgr2=0x55e928322600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 -- 172.21.15.37:0/3221410297 shutdown_connections 2024-04-26T02:01:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 -- 172.21.15.37:0/3221410297 wait complete. 2024-04-26T02:01:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f11d59ba740 1 -- start start 2024-04-26T02:01:17.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 Processor -- start 2024-04-26T02:01:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 -- 172.21.15.37:0/3884569747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 msgr2=0x7f9d6806abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 --2- 172.21.15.37:0/3884569747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 0x7f9d6806abc0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009990 tx=0x7f9d54009ca0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 -- 172.21.15.37:0/3884569747 shutdown_connections 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 --2- 172.21.15.37:0/3884569747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 0x7f9d68108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 --2- 172.21.15.37:0/3884569747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6810f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 --2- 172.21.15.37:0/3884569747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 0x7f9d6806abc0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.896+0000 7f9d6f400700 1 -- 172.21.15.37:0/3884569747 >> 172.21.15.37:0/3884569747 conn(0x7f9d680788e0 msgr2=0x7f9d68078cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f9d6f400700 1 -- 172.21.15.37:0/3884569747 shutdown_connections 2024-04-26T02:01:17.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f9d6f400700 1 -- 172.21.15.37:0/3884569747 wait complete. 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e928338390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e9283388d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928338e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1aea10 con 0x55e928332a00 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1afe80 con 0x55e928327820 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc1af0e0 con 0x55e928328140 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.897+0000 7f11d59ba740 10 client.0 _ll_register_callbacks cb 0x7f11bc0b9270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928338e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e9283388d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11d0a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928338e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46558/0 (socket says 172.21.15.37:46558) 2024-04-26T02:01:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11d0a00640 1 -- 172.21.15.37:0/3883956479 learned_addr learned my addr 172.21.15.37:0/3883956479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e9283388d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35730/0 (socket says 172.21.15.37:35730) 2024-04-26T02:01:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cbe00640 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e928338390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f9d6f400700 1 Processor -- start 2024-04-26T02:01:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 -- 172.21.15.37:0/3883956479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 msgr2=0x55e928338390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e928338390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 -- 172.21.15.37:0/3883956479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 msgr2=0x55e928338e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928338e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f9d6f400700 1 -- start start 2024-04-26T02:01:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.898+0000 7f11cb400640 1 -- 172.21.15.37:0/3883956479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9283a8310 con 0x55e928328140 2024-04-26T02:01:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f11cb400640 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e9283388d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55e928328a20 tx=0x7f11b80093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f41d88a7ad94250 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f11b5600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:01:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6f400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 0x7f9d6819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6f400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6819b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6f400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 0x7f9d6819fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6f400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68112c60 con 0x7f9d68108a60 2024-04-26T02:01:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6f400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68112ae0 con 0x7f9d6806a740 2024-04-26T02:01:17.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6f400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68112de0 con 0x7f9d6806b100 2024-04-26T02:01:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f11b5600640 1 -- 172.21.15.37:0/3883956479 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8033a60 con 0x55e928328140 2024-04-26T02:01:17.902 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f11b5600640 1 -- 172.21.15.37:0/3883956479 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f11b8033c00 con 0x55e928328140 2024-04-26T02:01:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d5fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6819b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d5fe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6819b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35744/0 (socket says 172.21.15.37:35744) 2024-04-26T02:01:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d5fe00700 1 -- 172.21.15.37:0/9014594 learned_addr learned my addr 172.21.15.37:0/9014594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f9d6d400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 0x7f9d6819fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.903 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.899+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11c401ca30 con 0x55e928328140 2024-04-26T02:01:17.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d5fe00700 1 -- 172.21.15.37:0/9014594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 msgr2=0x7f9d6819b2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T02:01:17.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 0x7f9d6819b2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d5fe00700 1 -- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 msgr2=0x7f9d6819fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 0x7f9d6819fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d5fe00700 1 -- 172.21.15.37:0/9014594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d681a02f0 con 0x7f9d6806b100 2024-04-26T02:01:17.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d6d400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 0x7f9d6819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:17.904 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f11b5600640 1 -- 172.21.15.37:0/3883956479 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f11b8033e20 con 0x55e928328140 2024-04-26T02:01:17.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6819b820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9d54000c00 tx=0x7f9d54009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f11d59ba740 10 client.25495 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:17.905 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55e9283a9560 con 0x55e928328140 2024-04-26T02:01:17.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d66000700 1 -- 172.21.15.37:0/9014594 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54027070 con 0x7f9d6806b100 2024-04-26T02:01:17.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d66000700 1 -- 172.21.15.37:0/9014594 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d54022470 con 0x7f9d6806b100 2024-04-26T02:01:17.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d66000700 1 -- 172.21.15.37:0/9014594 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d54017650 con 0x7f9d6806b100 2024-04-26T02:01:17.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.900+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d681a0580 con 0x7f9d6806b100 2024-04-26T02:01:17.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.901+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a0950 con 0x7f9d6806b100 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.901+0000 7f11b5600640 1 -- 172.21.15.37:0/3883956479 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11c401ca30 con 0x55e928328140 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11b5600640 1 -- 172.21.15.37:0/3883956479 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f11b803a070 con 0x55e928328140 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11d59ba740 10 client.25495 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11d59ba740 10 client.25495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11d59ba740 20 client.25495 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e9283a9ac0 0x55e9283abee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11d59ba740 4 client.25495 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e92829b320 con 0x55e9283a9ac0 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.902+0000 7f11cbe00640 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e9283a9ac0 0x55e9283abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.903+0000 7f11b5600640 10 client.25495 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.907 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.903+0000 7f11cbe00640 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e9283a9ac0 0x55e9283abee0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f41efddf6ce28da6 server_cookie=f916ec83bbabbbcc in_seq=0 out_seq=0 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.904+0000 7f9d66000700 1 -- 172.21.15.37:0/9014594 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9d5402a070 con 0x7f9d6806b100 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.905+0000 7f9d79d76b80 1 Processor -- start 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.905+0000 7f9d66000700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9d4c0a3f50 0x7f9d4c0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 -- start start 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d66000700 1 -- 172.21.15.37:0/9014594 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9d540d1470 con 0x7f9d6806b100 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d6d400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9d4c0a3f50 0x7f9d4c0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4e8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5597dc42e020 con 0x5597dc4e6560 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5597dc3d48c0 con 0x5597dc4dadd0 2024-04-26T02:01:17.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5597dc429fe0 con 0x5597dc4db6f0 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d6ca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d6ca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4db1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:33982/0 (socket says 172.21.15.37:33982) 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.906+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1718300268 learned_addr learned my addr 172.21.15.37:0/1718300268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d6d400700 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4e8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1718300268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 msgr2=0x5597dc4e6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1718300268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 msgr2=0x5597dc4e8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4e8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.907+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1718300268 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597dc3ca810 con 0x5597dc4dadd0 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.908+0000 7f9d6d400700 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.908+0000 7f9d6d400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9d4c0a3f50 0x7f9d4c0a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d60007770 tx=0x7f9d60007b30 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.908+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4db1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009be0 tx=0x7f9d58009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7681217a559fa33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.909+0000 7f9d5e000700 1 -- 172.21.15.37:0/1718300268 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5800ebb0 con 0x5597dc4dadd0 2024-04-26T02:01:17.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.909+0000 7f9d5e000700 1 -- 172.21.15.37:0/1718300268 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5800ed50 con 0x5597dc4dadd0 2024-04-26T02:01:17.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.909+0000 7f9d5e000700 1 -- 172.21.15.37:0/1718300268 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58010430 con 0x5597dc4dadd0 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.910+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1718300268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 msgr2=0x5597dc4db1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.910+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4db1b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009be0 tx=0x7f9d58009ef0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1718300268 shutdown_connections 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4e8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4db1b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1718300268 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1718300268 >> 172.21.15.37:0/1718300268 conn(0x5597dc40a0e0 msgr2=0x5597dc4d72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1718300268 shutdown_connections 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.911+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1718300268 wait complete. 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.912+0000 7f9d79d76b80 1 -- start start 2024-04-26T02:01:17.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.912+0000 7f9d79d76b80 1 Processor -- start 2024-04-26T02:01:17.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 0x5597dc4e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4e6560 0x5597dc4e3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5597dc4e8ff0 con 0x5597dc4dadd0 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5597dc3e79f0 con 0x5597dc4e6560 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5597dc4e9170 con 0x5597dc4db6f0 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d79d76b80 10 client.0 _ll_register_callbacks cb 0x5597dc3c68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d6ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 0x5597dc4e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d6ca00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 0x5597dc4e5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46580/0 (socket says 172.21.15.37:46580) 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d6ca00700 1 -- 172.21.15.37:0/254994998 learned_addr learned my addr 172.21.15.37:0/254994998 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.913+0000 7f9d6d400700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4e6560 0x5597dc4e3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d6ca00700 1 -- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 msgr2=0x5597dc4e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d6ca00700 1 -- 172.21.15.37:0/254994998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4e6560 msgr2=0x5597dc4e3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4e6560 0x5597dc4e3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d6ca00700 1 -- 172.21.15.37:0/254994998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597dc37ba60 con 0x5597dc4dadd0 2024-04-26T02:01:17.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 0x5597dc4e5ce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009310 tx=0x7f9d580094a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c064b74126f2e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f9d52a00700 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:17.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f11b5600640 1 -- 172.21.15.37:0/3883956479 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e92829b320 con 0x55e9283a9ac0 2024-04-26T02:01:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f11b5600640 10 client.25495 handle_command_reply: tid=0 2024-04-26T02:01:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.914+0000 7f11d59ba740 1 client.25495 shutdown 2024-04-26T02:01:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 msgr2=0x55e9283388d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e9283388d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55e928328a20 tx=0x7f11b80093a0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 shutdown_connections 2024-04-26T02:01:17.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55e9283a9ac0 0x55e9283abee0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d52a00700 1 -- 172.21.15.37:0/254994998 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58025020 con 0x5597dc4dadd0 2024-04-26T02:01:17.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d52a00700 1 -- 172.21.15.37:0/254994998 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9d58005580 con 0x5597dc4dadd0 2024-04-26T02:01:17.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5800ed50 con 0x5597dc4dadd0 2024-04-26T02:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d52a00700 1 -- 172.21.15.37:0/254994998 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9d58030050 con 0x5597dc4dadd0 2024-04-26T02:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d79d76b80 10 client.15351 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5597dc3c4c80 con 0x5597dc4dadd0 2024-04-26T02:01:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e928332a00 0x55e928338e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55e928328140 0x55e9283388d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 --2- 172.21.15.37:0/3883956479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55e928327820 0x55e928338390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11b5600640 0 client.25495 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 >> 172.21.15.37:0/3883956479 conn(0x55e928323d30 msgr2=0x55e9283301c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 shutdown_connections 2024-04-26T02:01:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f11d59ba740 1 -- 172.21.15.37:0/3883956479 wait complete. 2024-04-26T02:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.915+0000 7f9d52a00700 1 -- 172.21.15.37:0/254994998 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d5800ed50 con 0x5597dc4dadd0 2024-04-26T02:01:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d52a00700 1 -- 172.21.15.37:0/254994998 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9d58004d00 con 0x5597dc4dadd0 2024-04-26T02:01:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:01:17.918 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:01:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:01:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:01:17.919 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f11d59ba740 20 client.25495 trim_cache size 0 max 16384 2024-04-26T02:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d79d76b80 10 client.15351 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d79d76b80 10 client.15351 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d79d76b80 20 client.15351 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc4f5cb0 0x5597dc4f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d79d76b80 4 client.15351 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.916+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597dc43b810 con 0x5597dc4f5cb0 2024-04-26T02:01:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f11b00a3c20 msgr2=0x7f11b00a60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f11b00a3c20 0x7f11b00a60e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f11c40099a0 tx=0x7f11c4009500 comp rx=0 tx=0).stop 2024-04-26T02:01:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 msgr2=0x7f11cc1a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 0x7f11cc1a06c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f11bc005ab0 tx=0x7f11bc003c10 comp rx=0 tx=0).stop 2024-04-26T02:01:17.920 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 shutdown_connections 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f11b00a3c20 0x7f11b00a60e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f11cc10c590 0x7f11cc19a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11cc103ae0 0x7f11cc1a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 --2- 172.21.15.37:0/2240703458 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f11cc1031c0 0x7f11cc1a06c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 >> 172.21.15.37:0/2240703458 conn(0x7f11cc077f70 msgr2=0x7f11cc108510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc4f5cb0 0x5597dc4f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 shutdown_connections 2024-04-26T02:01:17.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:17.917+0000 7f11d2a00640 1 -- 172.21.15.37:0/2240703458 wait complete. 2024-04-26T02:01:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.918+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc4f5cb0 0x5597dc4f80a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2d389e050b7358d server_cookie=addc0313d72edf4f in_seq=0 out_seq=0 2024-04-26T02:01:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.918+0000 7f9d52a00700 10 client.15351 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.918+0000 7f9d52a00700 1 -- 172.21.15.37:0/254994998 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5597dc43b810 con 0x5597dc4f5cb0 2024-04-26T02:01:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.918+0000 7f9d52a00700 10 client.15351 handle_command_reply: tid=0 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 client.15351 shutdown 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 msgr2=0x5597dc4e5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 0x5597dc4e5ce0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009310 tx=0x7f9d580094a0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 shutdown_connections 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc4f5cb0 0x5597dc4f80a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4dadd0 0x5597dc4e5ce0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4e6560 0x5597dc4e3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/254994998 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4db6f0 0x5597dc4e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 >> 172.21.15.37:0/254994998 conn(0x5597dc40a0e0 msgr2=0x5597dc4d6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.919+0000 7f9d52a00700 0 client.15351 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.920+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 shutdown_connections 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.920+0000 7f9d79d76b80 1 -- 172.21.15.37:0/254994998 wait complete. 2024-04-26T02:01:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.920+0000 7f9d79d76b80 20 client.15351 trim_cache size 0 max 16384 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.941+0000 7f9d79d76b80 1 Processor -- start 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 -- start start 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4e34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 0x5597dc4fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4e6560 0x5597dc4fdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ad480 con 0x5597dc4db6f0 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681afd00 con 0x5597dc4dadd0 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681adaf0 con 0x5597dc4e6560 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d5fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 0x5597dc4fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d5fe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 0x5597dc4fd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46586/0 (socket says 172.21.15.37:46586) 2024-04-26T02:01:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.942+0000 7f9d5fe00700 1 -- 172.21.15.37:0/237729046 learned_addr learned my addr 172.21.15.37:0/237729046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4e34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d6d400700 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4e6560 0x5597dc4fdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5fe00700 1 -- 172.21.15.37:0/237729046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4e6560 msgr2=0x5597dc4fdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4e6560 0x5597dc4fdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5fe00700 1 -- 172.21.15.37:0/237729046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 msgr2=0x5597dc4e34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4e34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5fe00700 1 -- 172.21.15.37:0/237729046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597dc4f8550 con 0x5597dc4db6f0 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 0x5597dc4fd790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d54006010 tx=0x7f9d5401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0364036b2a070e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.943+0000 7f9d5d600700 1 -- 172.21.15.37:0/237729046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5409d470 con 0x5597dc4db6f0 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d5d600700 1 -- 172.21.15.37:0/237729046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d540a1820 con 0x5597dc4db6f0 2024-04-26T02:01:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d5d600700 1 -- 172.21.15.37:0/237729046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d540cee10 con 0x5597dc4db6f0 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 -- 172.21.15.37:0/237729046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 msgr2=0x5597dc4fd790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 0x5597dc4fd790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d54006010 tx=0x7f9d5401a040 comp rx=0 tx=0).stop 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 -- 172.21.15.37:0/237729046 shutdown_connections 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4e6560 0x5597dc4fdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4dadd0 0x5597dc4e34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/237729046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4db6f0 0x5597dc4fd790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.944+0000 7f9d79d76b80 1 -- 172.21.15.37:0/237729046 >> 172.21.15.37:0/237729046 conn(0x5597dc40a0e0 msgr2=0x5597dc4e2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.945+0000 7f9d79d76b80 1 -- 172.21.15.37:0/237729046 shutdown_connections 2024-04-26T02:01:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.945+0000 7f9d79d76b80 1 -- 172.21.15.37:0/237729046 wait complete. 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.945+0000 7f9d79d76b80 1 -- start start 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.945+0000 7f9d79d76b80 1 Processor -- start 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 0x5597dc4ee8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4db6f0 0x5597dc4ecc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ae1c0 con 0x5597dc4e6560 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681af630 con 0x5597dc4db6f0 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681ae890 con 0x5597dc4dadd0 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d79d76b80 10 client.0 _ll_register_callbacks cb 0x5597dc3c68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d6ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 0x5597dc4ee8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d6ca00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 0x5597dc4ee8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35794/0 (socket says 172.21.15.37:35794) 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d6d400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1807955010 learned_addr learned my addr 172.21.15.37:0/1807955010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.946+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4db6f0 0x5597dc4ecc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1807955010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4db6f0 msgr2=0x5597dc4ecc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4db6f0 0x5597dc4ecc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1807955010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 msgr2=0x5597dc4ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4ed2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d6ca00700 1 -- 172.21.15.37:0/1807955010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597dc4edab0 con 0x5597dc4dadd0 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4db6f0 0x5597dc4ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d6ca00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 0x5597dc4ee8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009310 tx=0x7f9d580038c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfcf1709aaaa3cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d52a00700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:01:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d52a00700 1 -- 172.21.15.37:0/1807955010 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58039060 con 0x5597dc4dadd0 2024-04-26T02:01:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d540a1820 con 0x5597dc4dadd0 2024-04-26T02:01:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.947+0000 7f9d52a00700 1 -- 172.21.15.37:0/1807955010 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9d58003a40 con 0x5597dc4dadd0 2024-04-26T02:01:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d52a00700 1 -- 172.21.15.37:0/1807955010 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9d58025040 con 0x5597dc4dadd0 2024-04-26T02:01:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 10 client.25507 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:17.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5597dc55d130 con 0x5597dc4dadd0 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d52a00700 1 -- 172.21.15.37:0/1807955010 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d540a1820 con 0x5597dc4dadd0 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d52a00700 1 -- 172.21.15.37:0/1807955010 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9d58010430 con 0x5597dc4dadd0 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 10 client.25507 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 10 client.25507 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 20 client.25507 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc55d7f0 0x5597dc55fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 4 client.25507 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T02:01:17.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.948+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x5597dc4358b0 con 0x5597dc55d7f0 2024-04-26T02:01:17.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.949+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc55d7f0 0x5597dc55fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:17.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.949+0000 7f9d5fe00700 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc55d7f0 0x5597dc55fc10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b029cd8a52f50c server_cookie=eb193cf65f81860b in_seq=0 out_seq=0 2024-04-26T02:01:17.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.949+0000 7f9d52a00700 10 client.25507 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.955+0000 7f9d52a00700 1 -- 172.21.15.37:0/1807955010 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5597dc4358b0 con 0x5597dc55d7f0 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.955+0000 7f9d52a00700 10 client.25507 handle_command_reply: tid=0 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 client.25507 shutdown 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 msgr2=0x5597dc4ee8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 0x5597dc4ee8c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009310 tx=0x7f9d580038c0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 shutdown_connections 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5597dc55d7f0 0x5597dc55fc10 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5597dc4e6560 0x5597dc4ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5597dc4dadd0 0x5597dc4ee8c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 --2- 172.21.15.37:0/1807955010 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5597dc4db6f0 0x5597dc4ecc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 >> 172.21.15.37:0/1807955010 conn(0x5597dc40a0e0 msgr2=0x5597dc4e4810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d52a00700 0 client.25507 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 shutdown_connections 2024-04-26T02:01:17.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 1 -- 172.21.15.37:0/1807955010 wait complete. 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.956+0000 7f9d79d76b80 20 client.25507 trim_cache size 0 max 16384 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 10, 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "118080ef": { 2024-04-26T02:01:17.957 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 10, 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:01:17.958 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:17.959 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:17.959 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:01:17.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9d4c0a3f50 msgr2=0x7f9d4c0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9d4c0a3f50 0x7f9d4c0a6410 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9d60007770 tx=0x7f9d60007b30 comp rx=0 tx=0).stop 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 msgr2=0x7f9d6819b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6819b820 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9d54000c00 tx=0x7f9d54009510 comp rx=0 tx=0).stop 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 shutdown_connections 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d68108a60 0x7f9d6819fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9d4c0a3f50 0x7f9d4c0a6410 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f9d6806a740 0x7f9d6819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 --2- 172.21.15.37:0/9014594 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f9d6806b100 0x7f9d6819b820 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.957+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 >> 172.21.15.37:0/9014594 conn(0x7f9d680788e0 msgr2=0x7f9d6810dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.958+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 shutdown_connections 2024-04-26T02:01:17.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:17.958+0000 7f9d6f400700 1 -- 172.21.15.37:0/9014594 wait complete. 2024-04-26T02:01:18.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:18 smithi037 ceph-mon[27906]: pgmap v103: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 59 MiB/s wr, 17 op/s 2024-04-26T02:01:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:18 smithi071 ceph-mon[32578]: pgmap v103: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 59 MiB/s wr, 17 op/s 2024-04-26T02:01:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:18 smithi045 ceph-mon[32520]: pgmap v103: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 59 MiB/s wr, 17 op/s 2024-04-26T02:01:18.818 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '118080ef', quiesce took 9.1 seconds. Will release after: 30.0 2024-04-26T02:01:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:20 smithi071 ceph-mon[32578]: pgmap v104: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 151 MiB/s wr, 41 op/s 2024-04-26T02:01:20.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:20 smithi037 ceph-mon[27906]: pgmap v104: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 151 MiB/s wr, 41 op/s 2024-04-26T02:01:20.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:20 smithi045 ceph-mon[32520]: pgmap v104: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 151 MiB/s wr, 41 op/s 2024-04-26T02:01:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:22 smithi071 ceph-mon[32578]: pgmap v105: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 151 MiB/s wr, 39 op/s 2024-04-26T02:01:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:22 smithi037 ceph-mon[27906]: pgmap v105: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 151 MiB/s wr, 39 op/s 2024-04-26T02:01:22.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:22 smithi045 ceph-mon[32520]: pgmap v105: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 151 MiB/s wr, 39 op/s 2024-04-26T02:01:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:24 smithi071 ceph-mon[32578]: pgmap v106: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 151 MiB/s wr, 43 op/s 2024-04-26T02:01:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:24 smithi037 ceph-mon[27906]: pgmap v106: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 151 MiB/s wr, 43 op/s 2024-04-26T02:01:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:24 smithi045 ceph-mon[32520]: pgmap v106: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 151 MiB/s wr, 43 op/s 2024-04-26T02:01:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:26 smithi071 ceph-mon[32578]: pgmap v107: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 92 MiB/s wr, 27 op/s 2024-04-26T02:01:26.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:26 smithi037 ceph-mon[27906]: pgmap v107: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 92 MiB/s wr, 27 op/s 2024-04-26T02:01:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:26 smithi045 ceph-mon[32520]: pgmap v107: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 92 MiB/s wr, 27 op/s 2024-04-26T02:01:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:28 smithi071 ceph-mon[32578]: pgmap v108: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 92 MiB/s wr, 27 op/s 2024-04-26T02:01:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:28 smithi037 ceph-mon[27906]: pgmap v108: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 92 MiB/s wr, 27 op/s 2024-04-26T02:01:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:28 smithi045 ceph-mon[32520]: pgmap v108: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 92 MiB/s wr, 27 op/s 2024-04-26T02:01:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:30 smithi071 ceph-mon[32578]: pgmap v109: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 92 MiB/s wr, 28 op/s 2024-04-26T02:01:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:30 smithi037 ceph-mon[27906]: pgmap v109: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 92 MiB/s wr, 28 op/s 2024-04-26T02:01:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:30 smithi045 ceph-mon[32520]: pgmap v109: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 92 MiB/s wr, 28 op/s 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 -- 172.21.15.37:0/3868649005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd89010e6f0 msgr2=0x7fd890110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 --2- 172.21.15.37:0/3868649005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd89010e6f0 0x7fd890110ae0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ca0 tx=0x7fd88802a8c0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 -- 172.21.15.37:0/3868649005 shutdown_connections 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 --2- 172.21.15.37:0/3868649005 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd89010e6f0 0x7fd890110ae0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 --2- 172.21.15.37:0/3868649005 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd8901014f0 0x7fd89010e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 --2- 172.21.15.37:0/3868649005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd890100b40 0x7fd890100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 -- 172.21.15.37:0/3868649005 >> 172.21.15.37:0/3868649005 conn(0x7fd8900fc750 msgr2=0x7fd8900feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 -- 172.21.15.37:0/3868649005 shutdown_connections 2024-04-26T02:01:31.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.066+0000 7fd894c00640 1 -- 172.21.15.37:0/3868649005 wait complete. 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.067+0000 7fd894c00640 1 Processor -- start 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.067+0000 7fd894c00640 1 -- start start 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd894c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890100b40 0x7fd8901a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd894c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 0x7fd8901a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd894c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89010e6f0 0x7fd89019a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd894c00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901141d0 con 0x7fd8901014f0 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd894c00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd890114050 con 0x7fd89010e6f0 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd894c00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd890114350 con 0x7fd890100b40 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 0x7fd8901a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd88f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890100b40 0x7fd8901a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd88f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890100b40 0x7fd8901a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37924/0 (socket says 172.21.15.37:37924) 2024-04-26T02:01:31.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 0x7fd8901a0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49538/0 (socket says 172.21.15.37:49538) 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd88f400640 1 -- 172.21.15.37:0/3850082900 learned_addr learned my addr 172.21.15.37:0/3850082900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.068+0000 7fd88fe00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89010e6f0 0x7fd89019a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd88ea00640 1 -- 172.21.15.37:0/3850082900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890100b40 msgr2=0x7fd8901a0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd88ea00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890100b40 0x7fd8901a0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd88ea00640 1 -- 172.21.15.37:0/3850082900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89010e6f0 msgr2=0x7fd89019a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd88ea00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89010e6f0 0x7fd89019a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd88ea00640 1 -- 172.21.15.37:0/3850082900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89019ad70 con 0x7fd8901014f0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd88ea00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 0x7fd8901a0980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9f0 tx=0x7fd87c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd887e00640 1 -- 172.21.15.37:0/3850082900 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c00ccc0 con 0x7fd8901014f0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd887e00640 1 -- 172.21.15.37:0/3850082900 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd87c0108e0 con 0x7fd8901014f0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89019b060 con 0x7fd8901014f0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd887e00640 1 -- 172.21.15.37:0/3850082900 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c018880 con 0x7fd8901014f0 2024-04-26T02:01:31.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.069+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8901a7300 con 0x7fd8901014f0 2024-04-26T02:01:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.074+0000 7fd898f82740 1 Processor -- start 2024-04-26T02:01:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.074+0000 7fd898f82740 1 -- start start 2024-04-26T02:01:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.074+0000 7fd887e00640 1 -- 172.21.15.37:0/3850082900 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd87c010430 con 0x7fd8901014f0 2024-04-26T02:01:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd898f82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba19b0 0x55abd2ba1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd898f82740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 0x55abd2bac650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd898f82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2baef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd898f82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abd2b07900 con 0x55abd2bacb90 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd898f82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55abd2aadef0 con 0x55abd2ba19b0 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd898f82740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55abd2b038c0 con 0x55abd2ba22d0 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2baef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 0x55abd2bac650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 0x55abd2bac650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37928/0 (socket says 172.21.15.37:37928) 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2baef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49554/0 (socket says 172.21.15.37:49554) 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba19b0 0x55abd2ba1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 -- 172.21.15.37:0/1019320440 learned_addr learned my addr 172.21.15.37:0/1019320440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:31.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 -- 172.21.15.37:0/1019320440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba19b0 msgr2=0x55abd2ba1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba19b0 0x55abd2ba1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 -- 172.21.15.37:0/1019320440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 msgr2=0x55abd2baef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2baef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.075+0000 7fd88f400640 1 -- 172.21.15.37:0/1019320440 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd281a210 con 0x55abd2ba22d0 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.076+0000 7fd88f400640 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 0x55abd2bac650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b7e0 tx=0x7fd88000bca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b17a0587be1cf48d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.076+0000 7fd87be00640 1 -- 172.21.15.37:0/1019320440 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd880003dd0 con 0x55abd2ba22d0 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.076+0000 7fd87be00640 1 -- 172.21.15.37:0/1019320440 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd880015070 con 0x55abd2ba22d0 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.076+0000 7fd87be00640 1 -- 172.21.15.37:0/1019320440 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8800106d0 con 0x55abd2ba22d0 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.076+0000 7fd887e00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd8740a3d00 0x7fd8740a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.076+0000 7fd887e00640 1 -- 172.21.15.37:0/3850082900 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fd87c0d0ab0 con 0x7fd8901014f0 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd88fe00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd8740a3d00 0x7fd8740a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 -- 172.21.15.37:0/1019320440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 msgr2=0x55abd2bac650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 0x55abd2bac650 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd88000b7e0 tx=0x7fd88000bca0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 -- 172.21.15.37:0/1019320440 shutdown_connections 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2baef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba22d0 0x55abd2bac650 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 --2- 172.21.15.37:0/1019320440 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba19b0 0x55abd2ba1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 -- 172.21.15.37:0/1019320440 >> 172.21.15.37:0/1019320440 conn(0x55abd2b9dec0 msgr2=0x55abd2b9e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 -- 172.21.15.37:0/1019320440 shutdown_connections 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd898f82740 1 -- 172.21.15.37:0/1019320440 wait complete. 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.077+0000 7fd88fe00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd8740a3d00 0x7fd8740a61c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd8880099a0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.078+0000 7fd898f82740 1 -- start start 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.078+0000 7fd898f82740 1 Processor -- start 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.078+0000 7fd898f82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2ba19b0 0x55abd2ba9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd898f82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2baa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd898f82740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2bacb90 0x55abd2baa580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd898f82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abd2baf7c0 con 0x55abd2ba19b0 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd898f82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55abd2ac1310 con 0x55abd2ba22d0 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd898f82740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55abd2baf940 con 0x55abd2bacb90 2024-04-26T02:01:31.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd898f82740 10 client.0 _ll_register_callbacks cb 0x55abd2a8fcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2ba19b0 0x55abd2ba9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2baa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2ba19b0 0x55abd2ba9b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49568/0 (socket says 172.21.15.37:49568) 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88f400640 1 -- 172.21.15.37:0/3620088651 learned_addr learned my addr 172.21.15.37:0/3620088651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2baa040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:46740/0 (socket says 172.21.15.37:46740) 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88fe00640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2bacb90 0x55abd2baa580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 -- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2bacb90 msgr2=0x55abd2baa580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2bacb90 0x55abd2baa580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 -- 172.21.15.37:0/3620088651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2ba19b0 msgr2=0x55abd2ba9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2ba19b0 0x55abd2ba9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88ea00640 1 -- 172.21.15.37:0/3620088651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd28e6620 con 0x55abd2ba22d0 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.079+0000 7fd88fe00640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2bacb90 0x55abd2baa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:31.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd88ea00640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2baa040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0ce1a0 tx=0x7fd87c00ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ab747328292d0f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd878c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd878c00640 1 -- 172.21.15.37:0/3620088651 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd87c09c610 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd878c00640 1 -- 172.21.15.37:0/3620088651 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fd87c0ce450 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd880015070 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd878c00640 1 -- 172.21.15.37:0/3620088651 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd87c09c430 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd898f82740 10 client.25454 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55abd2bbba40 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd878c00640 1 -- 172.21.15.37:0/3620088651 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd880015070 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.080+0000 7fd878c00640 1 -- 172.21.15.37:0/3620088651 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fd87c0a5200 con 0x55abd2ba22d0 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd898f82740 10 client.25454 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd898f82740 10 client.25454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd898f82740 20 client.25454 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd898f82740 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2bbcf00 0x55abd2bbf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd898f82740 4 client.25454 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:31.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd2b1b8b0 con 0x55abd2bbcf00 2024-04-26T02:01:31.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.081+0000 7fd88f400640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2bbcf00 0x55abd2bbf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.082+0000 7fd88f400640 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2bbcf00 0x55abd2bbf310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97fafac6dce2b9f6 server_cookie=87264ae3a966666f in_seq=0 out_seq=0 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.082+0000 7fd878c00640 10 client.25454 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.082+0000 7fd878c00640 1 -- 172.21.15.37:0/3620088651 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abd2b1b8b0 con 0x55abd2bbcf00 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.082+0000 7fd878c00640 10 client.25454 handle_command_reply: tid=0 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 client.25454 shutdown 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 msgr2=0x55abd2baa040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2baa040 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd87c0ce1a0 tx=0x7fd87c00ce60 comp rx=0 tx=0).stop 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 shutdown_connections 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2bbcf00 0x55abd2bbf310 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2bacb90 0x55abd2baa580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2baa040 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd878c00640 0 client.25454 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 --2- 172.21.15.37:0/3620088651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2ba19b0 0x55abd2ba9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 >> 172.21.15.37:0/3620088651 conn(0x55abd2b9dec0 msgr2=0x55abd2ba3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 shutdown_connections 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 1 -- 172.21.15.37:0/3620088651 wait complete. 2024-04-26T02:01:31.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.083+0000 7fd898f82740 20 client.25454 trim_cache size 0 max 16384 2024-04-26T02:01:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 Processor -- start 2024-04-26T02:01:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 -- start start 2024-04-26T02:01:31.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2b9d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bc4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901adba0 con 0x55abd2bacb90 2024-04-26T02:01:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901b0420 con 0x55abd2ba22d0 2024-04-26T02:01:31.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd898f82740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901ae210 con 0x55abd2ba19b0 2024-04-26T02:01:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.100+0000 7fd88fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bc4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bc4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49582/0 (socket says 172.21.15.37:49582) 2024-04-26T02:01:31.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 -- 172.21.15.37:0/2532652810 learned_addr learned my addr 172.21.15.37:0/2532652810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88f400640 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88ea00640 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2b9d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 -- 172.21.15.37:0/2532652810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 msgr2=0x55abd2bb8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 -- 172.21.15.37:0/2532652810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 msgr2=0x55abd2b9d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2b9d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 -- 172.21.15.37:0/2532652810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd2baef80 con 0x55abd2bacb90 2024-04-26T02:01:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.101+0000 7fd88fe00640 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bc4c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd890101a90 tx=0x7fd888041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9955f2e4ce25ce06 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd87b400640 1 -- 172.21.15.37:0/2532652810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888056070 con 0x55abd2bacb90 2024-04-26T02:01:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd87b400640 1 -- 172.21.15.37:0/2532652810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd888036050 con 0x55abd2bacb90 2024-04-26T02:01:31.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd87b400640 1 -- 172.21.15.37:0/2532652810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88802f660 con 0x55abd2bacb90 2024-04-26T02:01:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 -- 172.21.15.37:0/2532652810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 msgr2=0x55abd2bc4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bc4c20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd890101a90 tx=0x7fd888041000 comp rx=0 tx=0).stop 2024-04-26T02:01:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 -- 172.21.15.37:0/2532652810 shutdown_connections 2024-04-26T02:01:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bc4c20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2b9d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 --2- 172.21.15.37:0/2532652810 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.102+0000 7fd898f82740 1 -- 172.21.15.37:0/2532652810 >> 172.21.15.37:0/2532652810 conn(0x55abd2b9dec0 msgr2=0x55abd2ba4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.103+0000 7fd898f82740 1 -- 172.21.15.37:0/2532652810 shutdown_connections 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.103+0000 7fd898f82740 1 -- 172.21.15.37:0/2532652810 wait complete. 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.103+0000 7fd898f82740 1 -- start start 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.103+0000 7fd898f82740 1 Processor -- start 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2bb2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bb26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901ae8e0 con 0x55abd2bacb90 2024-04-26T02:01:31.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901afd50 con 0x55abd2ba22d0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd8901aefb0 con 0x55abd2ba19b0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd898f82740 10 client.0 _ll_register_callbacks cb 0x7fd87c09c8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bb26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bb26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49592/0 (socket says 172.21.15.37:49592) 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 -- 172.21.15.37:0/373933691 learned_addr learned my addr 172.21.15.37:0/373933691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88f400640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88ea00640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2bb2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 -- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 msgr2=0x55abd2bb4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 -- 172.21.15.37:0/373933691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 msgr2=0x55abd2bb2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2bb2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.104+0000 7fd88fe00640 1 -- 172.21.15.37:0/373933691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd2bb2d10 con 0x55abd2bacb90 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd88fe00640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bb26a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ca0 tx=0x7fd888005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e3ce1fa6546e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:31.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd878c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd878c00640 1 -- 172.21.15.37:0/373933691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888056040 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd888036050 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd878c00640 1 -- 172.21.15.37:0/373933691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fd88802fc90 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd878c00640 1 -- 172.21.15.37:0/373933691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd888031bb0 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd898f82740 10 client.15375 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.105+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55abd2c23340 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd878c00640 1 -- 172.21.15.37:0/373933691 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd888036050 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd878c00640 1 -- 172.21.15.37:0/373933691 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fd888050070 con 0x55abd2bacb90 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd898f82740 10 client.15375 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd898f82740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd898f82740 20 client.15375 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd898f82740 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2c23a30 0x55abd2c25e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd898f82740 4 client.15375 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:31.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abd2b15320 con 0x55abd2c23a30 2024-04-26T02:01:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.106+0000 7fd88f400640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2c23a30 0x55abd2c25e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.107+0000 7fd88f400640 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2c23a30 0x55abd2c25e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972faf919aa29b2e server_cookie=1b08593b9613dcba in_seq=0 out_seq=0 2024-04-26T02:01:31.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.107+0000 7fd878c00640 10 client.15375 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.118+0000 7fd878c00640 1 -- 172.21.15.37:0/373933691 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abd2b15320 con 0x55abd2c23a30 2024-04-26T02:01:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.118+0000 7fd878c00640 10 client.15375 handle_command_reply: tid=0 2024-04-26T02:01:31.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.118+0000 7fd898f82740 1 client.15375 shutdown 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.118+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 msgr2=0x55abd2bb26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.118+0000 7fd898f82740 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bb26a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ca0 tx=0x7fd888005ef0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 shutdown_connections 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55abd2c23a30 0x55abd2c25e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abd2bacb90 0x55abd2bb26a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55abd2ba22d0 0x55abd2bb2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 --2- 172.21.15.37:0/373933691 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55abd2ba19b0 0x55abd2bb4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd878c00640 0 client.15375 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 >> 172.21.15.37:0/373933691 conn(0x55abd2b9dec0 msgr2=0x55abd2ba3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 shutdown_connections 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 1 -- 172.21.15.37:0/373933691 wait complete. 2024-04-26T02:01:31.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.119+0000 7fd898f82740 20 client.15375 trim_cache size 0 max 16384 2024-04-26T02:01:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:01:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:01:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:01:31.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd8740a3d00 msgr2=0x7fd8740a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd8740a3d00 0x7fd8740a61c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd888006010 tx=0x7fd8880099a0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 msgr2=0x7fd8901a0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 0x7fd8901a0980 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd87c00e9f0 tx=0x7fd87c00eeb0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 shutdown_connections 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd8740a3d00 0x7fd8740a61c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd89010e6f0 0x7fd89019a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd8901014f0 0x7fd8901a0980 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 --2- 172.21.15.37:0/3850082900 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd890100b40 0x7fd8901a0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 >> 172.21.15.37:0/3850082900 conn(0x7fd8900fc750 msgr2=0x7fd8900feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 shutdown_connections 2024-04-26T02:01:31.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:31.120+0000 7fd894c00640 1 -- 172.21.15.37:0/3850082900 wait complete. 2024-04-26T02:01:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:32 smithi071 ceph-mon[32578]: pgmap v110: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.0 KiB/s wr, 4 op/s 2024-04-26T02:01:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:32 smithi037 ceph-mon[27906]: pgmap v110: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.0 KiB/s wr, 4 op/s 2024-04-26T02:01:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:32 smithi045 ceph-mon[32520]: pgmap v110: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 8.0 KiB/s wr, 4 op/s 2024-04-26T02:01:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:34 smithi071 ceph-mon[32578]: pgmap v111: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:34 smithi037 ceph-mon[27906]: pgmap v111: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:34 smithi045 ceph-mon[32520]: pgmap v111: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 11 KiB/s wr, 8 op/s 2024-04-26T02:01:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:36 smithi071 ceph-mon[32578]: pgmap v112: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-26T02:01:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:36 smithi037 ceph-mon[27906]: pgmap v112: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-26T02:01:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:36 smithi045 ceph-mon[32520]: pgmap v112: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-26T02:01:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:38 smithi071 ceph-mon[32578]: pgmap v113: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-26T02:01:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:38 smithi037 ceph-mon[27906]: pgmap v113: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-26T02:01:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:38 smithi045 ceph-mon[32520]: pgmap v113: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-04-26T02:01:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:40 smithi071 ceph-mon[32578]: pgmap v114: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-26T02:01:40.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:40 smithi037 ceph-mon[27906]: pgmap v114: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-26T02:01:40.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:40 smithi045 ceph-mon[32520]: pgmap v114: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 5.7 KiB/s wr, 5 op/s 2024-04-26T02:01:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:42 smithi071 ceph-mon[32578]: pgmap v115: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-04-26T02:01:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:42 smithi037 ceph-mon[27906]: pgmap v115: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-04-26T02:01:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:42 smithi045 ceph-mon[32520]: pgmap v115: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 -- 172.21.15.37:0/3016306154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c100ed0 msgr2=0x7fde0c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 --2- 172.21.15.37:0/3016306154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c100ed0 0x7fde0c1012b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009900 tx=0x7fddfc02a400 comp rx=0 tx=0).stop 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 -- 172.21.15.37:0/3016306154 shutdown_connections 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 --2- 172.21.15.37:0/3016306154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c105fa0 0x7fde0c10db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 --2- 172.21.15.37:0/3016306154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c101880 0x7fde0c1058c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 --2- 172.21.15.37:0/3016306154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c100ed0 0x7fde0c1012b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 -- 172.21.15.37:0/3016306154 >> 172.21.15.37:0/3016306154 conn(0x7fde0c0fc730 msgr2=0x7fde0c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 -- 172.21.15.37:0/3016306154 shutdown_connections 2024-04-26T02:01:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.268+0000 7fde12800640 1 -- 172.21.15.37:0/3016306154 wait complete. 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.269+0000 7fde12800640 1 Processor -- start 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.269+0000 7fde12800640 1 -- start start 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde12800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 0x7fde0c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde12800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c105fa0 0x7fde0c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde12800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c19a640 0x7fde0c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde12800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c110f50 con 0x7fde0c100ed0 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde12800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c110dd0 con 0x7fde0c19a640 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde12800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1110d0 con 0x7fde0c105fa0 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 0x7fde0c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 0x7fde0c1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34180/0 (socket says 172.21.15.37:34180) 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde11400640 1 -- 172.21.15.37:0/4290568561 learned_addr learned my addr 172.21.15.37:0/4290568561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde11e00640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c19a640 0x7fde0c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.270+0000 7fde10a00640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c105fa0 0x7fde0c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11400640 1 -- 172.21.15.37:0/4290568561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c105fa0 msgr2=0x7fde0c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11400640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c105fa0 0x7fde0c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11400640 1 -- 172.21.15.37:0/4290568561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c19a640 msgr2=0x7fde0c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11400640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c19a640 0x7fde0c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11400640 1 -- 172.21.15.37:0/4290568561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c19b160 con 0x7fde0c100ed0 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11e00640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c19a640 0x7fde0c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde11400640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 0x7fde0c1a0440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fddfc005ab0 tx=0x7fddfc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde0a000640 1 -- 172.21.15.37:0/4290568561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc03f070 con 0x7fde0c100ed0 2024-04-26T02:01:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.271+0000 7fde0a000640 1 -- 172.21.15.37:0/4290568561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddfc031b60 con 0x7fde0c100ed0 2024-04-26T02:01:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.272+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c19b3f0 con 0x7fde0c100ed0 2024-04-26T02:01:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.272+0000 7fde0a000640 1 -- 172.21.15.37:0/4290568561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc035030 con 0x7fde0c100ed0 2024-04-26T02:01:44.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.272+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c1a7590 con 0x7fde0c100ed0 2024-04-26T02:01:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.275+0000 7fde0a000640 1 -- 172.21.15.37:0/4290568561 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fddfc042070 con 0x7fde0c100ed0 2024-04-26T02:01:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.275+0000 7fde16b8f740 1 Processor -- start 2024-04-26T02:01:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.275+0000 7fde16b8f740 1 -- start start 2024-04-26T02:01:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.275+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1e2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ed650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1eff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde16b8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fae148900 con 0x555fae1e29b0 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde16b8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555fae0eeef0 con 0x555fae1e32d0 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde16b8f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fae1448c0 con 0x555fae1edb90 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1e2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde11e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1e2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34186/0 (socket says 172.21.15.37:34186) 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde11e00640 1 -- 172.21.15.37:0/1229198076 learned_addr learned my addr 172.21.15.37:0/1229198076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde10a00640 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1eff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde11400640 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ed650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde10a00640 1 -- 172.21.15.37:0/1229198076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 msgr2=0x555fae1ed650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde10a00640 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ed650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde10a00640 1 -- 172.21.15.37:0/1229198076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 msgr2=0x555fae1e2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde10a00640 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1e2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde10a00640 1 -- 172.21.15.37:0/1229198076 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fade5b210 con 0x555fae1edb90 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.276+0000 7fde11e00640 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde0a000640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fddf00a3d00 0x7fddf00a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde10a00640 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1eff80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fde0c107a00 tx=0x7fddf8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af3a132ade9b4c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde0a000640 1 -- 172.21.15.37:0/4290568561 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fddfc0f0ae0 con 0x7fde0c100ed0 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde11e00640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fddf00a3d00 0x7fddf00a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde02600640 1 -- 172.21.15.37:0/1229198076 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8004250 con 0x555fae1edb90 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde02600640 1 -- 172.21.15.37:0/1229198076 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf8009e10 con 0x555fae1edb90 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.277+0000 7fde02600640 1 -- 172.21.15.37:0/1229198076 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8004cd0 con 0x555fae1edb90 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 -- 172.21.15.37:0/1229198076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 msgr2=0x555fae1eff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1eff80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fde0c107a00 tx=0x7fddf8011810 comp rx=0 tx=0).stop 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde11e00640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fddf00a3d00 0x7fddf00a61c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde04012bb0 tx=0x7fde04009500 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:44.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 -- 172.21.15.37:0/1229198076 shutdown_connections 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1eff80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ed650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1229198076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1e2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 -- 172.21.15.37:0/1229198076 >> 172.21.15.37:0/1229198076 conn(0x555fae1deec0 msgr2=0x555fae1df2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 -- 172.21.15.37:0/1229198076 shutdown_connections 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 -- 172.21.15.37:0/1229198076 wait complete. 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.278+0000 7fde16b8f740 1 -- start start 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 Processor -- start 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1eaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1e32d0 0x555fae1eafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1edb90 0x555fae1eb510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555fae1f07c0 con 0x555fae1e29b0 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555fae102310 con 0x555fae1edb90 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555fae1f0940 con 0x555fae1e32d0 2024-04-26T02:01:44.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde16b8f740 10 client.0 _ll_register_callbacks cb 0x555fae0d0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1eaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde10a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1e32d0 0x555fae1eafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde10a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1e32d0 0x555fae1eafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:40364/0 (socket says 172.21.15.37:40364) 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1eaa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34198/0 (socket says 172.21.15.37:34198) 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.279+0000 7fde10a00640 1 -- 172.21.15.37:0/3701967619 learned_addr learned my addr 172.21.15.37:0/3701967619 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11e00640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1edb90 0x555fae1eb510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11400640 1 -- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1e32d0 msgr2=0x555fae1eafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11400640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1e32d0 0x555fae1eafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11400640 1 -- 172.21.15.37:0/3701967619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1edb90 msgr2=0x555fae1eb510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11400640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1edb90 0x555fae1eb510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11400640 1 -- 172.21.15.37:0/3701967619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fadf27620 con 0x555fae1e29b0 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11e00640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1edb90 0x555fae1eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:44.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde11400640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1eaa90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0ee200 tx=0x7fddfc009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1088ebb2d8310d5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fddf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fddf6a00640 1 -- 172.21.15.37:0/3701967619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc049070 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fddf6a00640 1 -- 172.21.15.37:0/3701967619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fddfc03b460 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.280+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf8009e10 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fddf6a00640 1 -- 172.21.15.37:0/3701967619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fddfc0bc590 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 10 client.15387 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555fae1f07c0 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fddf6a00640 1 -- 172.21.15.37:0/3701967619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddf8009e10 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fddf6a00640 1 -- 172.21.15.37:0/3701967619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fddfc04c060 con 0x555fae1e29b0 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 10 client.15387 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 10 client.15387 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 20 client.15387 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae1fdd90 0x555fae2001c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 4 client.15387 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.281+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fae15c8b0 con 0x555fae1fdd90 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.282+0000 7fde10a00640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae1fdd90 0x555fae2001c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.282+0000 7fde10a00640 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae1fdd90 0x555fae2001c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=172d192346e6c830 server_cookie=e702ff8dcfa7067f in_seq=0 out_seq=0 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.282+0000 7fddf6a00640 10 client.15387 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.283+0000 7fddf6a00640 1 -- 172.21.15.37:0/3701967619 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fae15c8b0 con 0x555fae1fdd90 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.283+0000 7fddf6a00640 10 client.15387 handle_command_reply: tid=0 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.283+0000 7fde16b8f740 1 client.15387 shutdown 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 msgr2=0x555fae1eaa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1eaa90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0ee200 tx=0x7fddfc009560 comp rx=0 tx=0).stop 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 shutdown_connections 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae1fdd90 0x555fae2001c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1edb90 0x555fae1eb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1e32d0 0x555fae1eafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 --2- 172.21.15.37:0/3701967619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1eaa90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 >> 172.21.15.37:0/3701967619 conn(0x555fae1deec0 msgr2=0x555fae1e4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fddf6a00640 0 client.15387 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 shutdown_connections 2024-04-26T02:01:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 1 -- 172.21.15.37:0/3701967619 wait complete. 2024-04-26T02:01:44.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.284+0000 7fde16b8f740 20 client.15387 trim_cache size 0 max 16384 2024-04-26T02:01:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.300+0000 7fde16b8f740 1 Processor -- start 2024-04-26T02:01:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 -- start start 2024-04-26T02:01:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae205a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1adb00 con 0x555fae1e29b0 2024-04-26T02:01:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1b0380 con 0x555fae1e32d0 2024-04-26T02:01:44.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde16b8f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ae170 con 0x555fae1edb90 2024-04-26T02:01:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde11e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae205a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34214/0 (socket says 172.21.15.37:34214) 2024-04-26T02:01:44.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde11400640 1 -- 172.21.15.37:0/2669878479 learned_addr learned my addr 172.21.15.37:0/2669878479 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.301+0000 7fde10a00640 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde11400640 1 -- 172.21.15.37:0/2669878479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 msgr2=0x555fae205a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde11400640 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae205a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde11400640 1 -- 172.21.15.37:0/2669878479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 msgr2=0x555fae1ddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde11400640 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ddea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde11400640 1 -- 172.21.15.37:0/2669878479 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fae1e38c0 con 0x555fae1e29b0 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde10a00640 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde11400640 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f9680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0c5830 tx=0x7fddfc0baf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af82d5742f0ed0c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde01c00640 1 -- 172.21.15.37:0/2669878479 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc049070 con 0x555fae1e29b0 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde01c00640 1 -- 172.21.15.37:0/2669878479 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddfc0bd150 con 0x555fae1e29b0 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.302+0000 7fde01c00640 1 -- 172.21.15.37:0/2669878479 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc0bc3b0 con 0x555fae1e29b0 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 -- 172.21.15.37:0/2669878479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 msgr2=0x555fae1f9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f9680 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0c5830 tx=0x7fddfc0baf60 comp rx=0 tx=0).stop 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 -- 172.21.15.37:0/2669878479 shutdown_connections 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae205a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1ddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 --2- 172.21.15.37:0/2669878479 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f9680 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 -- 172.21.15.37:0/2669878479 >> 172.21.15.37:0/2669878479 conn(0x555fae1deec0 msgr2=0x555fae1dd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 -- 172.21.15.37:0/2669878479 shutdown_connections 2024-04-26T02:01:44.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 -- 172.21.15.37:0/2669878479 wait complete. 2024-04-26T02:01:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.303+0000 7fde16b8f740 1 -- start start 2024-04-26T02:01:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 Processor -- start 2024-04-26T02:01:44.305 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1ae840 con 0x555fae1e29b0 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1afcb0 con 0x555fae1e32d0 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c1aef10 con 0x555fae1edb90 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde16b8f740 10 client.0 _ll_register_callbacks cb 0x7fddfc005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde11400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34222/0 (socket says 172.21.15.37:34222) 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde11400640 1 -- 172.21.15.37:0/1055543395 learned_addr learned my addr 172.21.15.37:0/1055543395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.304+0000 7fde10a00640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11e00640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11400640 1 -- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 msgr2=0x555fae1f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11400640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.306 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11400640 1 -- 172.21.15.37:0/1055543395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 msgr2=0x555fae1f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11400640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11400640 1 -- 172.21.15.37:0/1055543395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fae2625c0 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde11400640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f2b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b9420 tx=0x7fddfc005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cfa911bd8f232 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fddf6a00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fddf6a00640 1 -- 172.21.15.37:0/1055543395 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc049070 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fddf6a00640 1 -- 172.21.15.37:0/1055543395 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fddfc0318c0 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.305+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddfc0bd150 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fddf6a00640 1 -- 172.21.15.37:0/1055543395 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fddfc0bc3b0 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 10 client.15393 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555fae2625c0 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fddf6a00640 1 -- 172.21.15.37:0/1055543395 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fddfc0bd150 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fddf6a00640 1 -- 172.21.15.37:0/1055543395 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fddfc0a7110 con 0x555fae1e29b0 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 10 client.15393 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 10 client.15393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 20 client.15393 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:44.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae263b80 0x555fae265f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 4 client.15393 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.306+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555fae156320 con 0x555fae263b80 2024-04-26T02:01:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.307+0000 7fde10a00640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae263b80 0x555fae265f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.307+0000 7fde10a00640 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae263b80 0x555fae265f70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fafa85c0bf06f95 server_cookie=95e4258313274629 in_seq=0 out_seq=0 2024-04-26T02:01:44.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.307+0000 7fddf6a00640 10 client.15393 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fddf6a00640 1 -- 172.21.15.37:0/1055543395 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555fae156320 con 0x555fae263b80 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fddf6a00640 10 client.15393 handle_command_reply: tid=0 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 client.15393 shutdown 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 msgr2=0x555fae1f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f2b60 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fddfc0b9420 tx=0x7fddfc005e80 comp rx=0 tx=0).stop 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 shutdown_connections 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555fae263b80 0x555fae265f70 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555fae1edb90 0x555fae1f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555fae1e32d0 0x555fae1f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 --2- 172.21.15.37:0/1055543395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555fae1e29b0 0x555fae1f2b60 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 >> 172.21.15.37:0/1055543395 conn(0x555fae1deec0 msgr2=0x555fae1ecb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fddf6a00640 0 client.15393 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 shutdown_connections 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 1 -- 172.21.15.37:0/1055543395 wait complete. 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.315+0000 7fde16b8f740 20 client.15393 trim_cache size 0 max 16384 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:01:44.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:01:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fddf00a3d00 msgr2=0x7fddf00a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fddf00a3d00 0x7fddf00a61c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fde04012bb0 tx=0x7fde04009500 comp rx=0 tx=0).stop 2024-04-26T02:01:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 msgr2=0x7fde0c1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 0x7fde0c1a0440 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fddfc005ab0 tx=0x7fddfc003c10 comp rx=0 tx=0).stop 2024-04-26T02:01:44.316 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 shutdown_connections 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fddf00a3d00 0x7fddf00a61c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fde0c19a640 0x7fde0c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fde0c105fa0 0x7fde0c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 --2- 172.21.15.37:0/4290568561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde0c100ed0 0x7fde0c1a0440 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.316+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 >> 172.21.15.37:0/4290568561 conn(0x7fde0c0fc730 msgr2=0x7fde0c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.317+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 shutdown_connections 2024-04-26T02:01:44.317 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:44.317+0000 7fde12800640 1 -- 172.21.15.37:0/4290568561 wait complete. 2024-04-26T02:01:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:44 smithi071 ceph-mon[32578]: pgmap v116: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-04-26T02:01:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:44 smithi037 ceph-mon[27906]: pgmap v116: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-04-26T02:01:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:44 smithi045 ceph-mon[32520]: pgmap v116: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 3.5 KiB/s wr, 4 op/s 2024-04-26T02:01:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:46 smithi071 ceph-mon[32578]: pgmap v117: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:01:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:46 smithi037 ceph-mon[27906]: pgmap v117: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:01:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:46 smithi045 ceph-mon[32520]: pgmap v117: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:01:48.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:48 smithi037 ceph-mon[27906]: pgmap v118: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:01:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:48 smithi071 ceph-mon[32578]: pgmap v118: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:01:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:48 smithi045 ceph-mon[32520]: pgmap v118: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:01:48.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:01:49.299 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:50 smithi071 ceph-mon[32578]: pgmap v119: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:50.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:50 smithi037 ceph-mon[27906]: pgmap v119: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:50 smithi045 ceph-mon[32520]: pgmap v119: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:51.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 -- 172.21.15.37:0/4124274703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54104ec0 msgr2=0x7f8e541052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 --2- 172.21.15.37:0/4124274703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54104ec0 0x7f8e541052a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8e440099c0 tx=0x7f8e44009cd0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 -- 172.21.15.37:0/4124274703 shutdown_connections 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 --2- 172.21.15.37:0/4124274703 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 0x7f8e54111b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 --2- 172.21.15.37:0/4124274703 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54104ec0 0x7f8e541052a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 --2- 172.21.15.37:0/4124274703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e54105870 0x7f8e541098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.694+0000 7f8e59000700 1 -- 172.21.15.37:0/4124274703 >> 172.21.15.37:0/4124274703 conn(0x7f8e54100750 msgr2=0x7f8e54102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.695+0000 7f8e59000700 1 -- 172.21.15.37:0/4124274703 shutdown_connections 2024-04-26T02:01:51.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.695+0000 7f8e59000700 1 -- 172.21.15.37:0/4124274703 wait complete. 2024-04-26T02:01:51.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.696+0000 7f8e59000700 1 Processor -- start 2024-04-26T02:01:51.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.696+0000 7f8e59000700 1 -- start start 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e59000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e54104ec0 0x7f8e5419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e59000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 0x7f8e5419b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e59000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 0x7f8e5419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e59000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54114e80 con 0x7f8e54104ec0 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e59000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54114d00 con 0x7f8e54109fa0 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e59000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54115000 con 0x7f8e54105870 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e51400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 0x7f8e5419b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e51400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 0x7f8e5419b910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44716/0 (socket says 172.21.15.37:44716) 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e51400700 1 -- 172.21.15.37:0/1484887917 learned_addr learned my addr 172.21.15.37:0/1484887917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:51.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e51e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e54104ec0 0x7f8e5419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:51.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.697+0000 7f8e52800700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 0x7f8e5419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:51.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e51400700 1 -- 172.21.15.37:0/1484887917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 msgr2=0x7f8e5419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:51.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e51400700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 0x7f8e5419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e51400700 1 -- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e54104ec0 msgr2=0x7f8e5419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e51400700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e54104ec0 0x7f8e5419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e51400700 1 -- 172.21.15.37:0/1484887917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e541a03e0 con 0x7f8e54105870 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e52800700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 0x7f8e5419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e51400700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 0x7f8e5419b910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d950 tx=0x7f8e4000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40016070 con 0x7f8e54105870 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e541a06d0 con 0x7f8e54105870 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e40004830 con 0x7f8e54105870 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.698+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e40005420 con 0x7f8e54105870 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.699+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e541a7f50 con 0x7f8e54105870 2024-04-26T02:01:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.700+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e54069730 con 0x7f8e54105870 2024-04-26T02:01:51.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.702+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e40004220 con 0x7f8e54105870 2024-04-26T02:01:51.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.703+0000 7f8e4aa00700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8e380a3e30 0x7f8e380a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.703+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8e400d0530 con 0x7f8e54105870 2024-04-26T02:01:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.703+0000 7f8e51e00700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8e380a3e30 0x7f8e380a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.704+0000 7f8e51e00700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8e380a3e30 0x7f8e380a62f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8e44003760 tx=0x7f8e44005fb0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:51.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.708+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e4001f020 con 0x7f8e54105870 2024-04-26T02:01:51.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.994+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e5419c730 con 0x7f8e54105870 2024-04-26T02:01:51.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.996+0000 7f8e4aa00700 1 -- 172.21.15.37:0/1484887917 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7f8e4009cba0 con 0x7f8e54105870 2024-04-26T02:01:51.995 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:01:51.995 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:01:51.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8e380a3e30 msgr2=0x7f8e380a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:51.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8e380a3e30 0x7f8e380a62f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8e44003760 tx=0x7f8e44005fb0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 msgr2=0x7f8e5419b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:51.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 0x7f8e5419b910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d950 tx=0x7f8e4000dc60 comp rx=0 tx=0).stop 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 shutdown_connections 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e54104ec0 0x7f8e5419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8e380a3e30 0x7f8e380a62f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8e54109fa0 0x7f8e5419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 --2- 172.21.15.37:0/1484887917 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8e54105870 0x7f8e5419b910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 >> 172.21.15.37:0/1484887917 conn(0x7f8e54100750 msgr2=0x7f8e54109130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 shutdown_connections 2024-04-26T02:01:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:51.999+0000 7f8e59000700 1 -- 172.21.15.37:0/1484887917 wait complete. 2024-04-26T02:01:51.999 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:01:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:52 smithi071 ceph-mon[32578]: pgmap v120: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:52 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1484887917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:52.403 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:52 smithi037 ceph-mon[27906]: pgmap v120: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:52.403 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:52 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1484887917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:52 smithi045 ceph-mon[32520]: pgmap v120: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:52 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1484887917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:01:52.873 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db --set-id 118080ef --release --await' 2024-04-26T02:01:52.874 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db --set-id 118080ef --release --await 2024-04-26T02:01:53.344 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:01:54.370 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:54 smithi037 ceph-mon[27906]: pgmap v121: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:54 smithi071 ceph-mon[32578]: pgmap v121: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:54 smithi045 ceph-mon[32520]: pgmap v121: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 -- 172.21.15.37:0/1808935836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4102950 msgr2=0x7fd2d4102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1808935836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4102950 0x7fd2d4102dd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098f0 tx=0x7fd2c0009c00 comp rx=0 tx=0).stop 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 -- 172.21.15.37:0/1808935836 shutdown_connections 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1808935836 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d4103310 0x7fd2d410f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1808935836 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4108950 0x7fd2d4108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1808935836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4102950 0x7fd2d4102dd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 -- 172.21.15.37:0/1808935836 >> 172.21.15.37:0/1808935836 conn(0x7fd2d40fe460 msgr2=0x7fd2d4100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 -- 172.21.15.37:0/1808935836 shutdown_connections 2024-04-26T02:01:55.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.767+0000 7fd2d9600700 1 -- 172.21.15.37:0/1808935836 wait complete. 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.768+0000 7fd2d9600700 1 Processor -- start 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.768+0000 7fd2d9600700 1 -- start start 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d9600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 0x7fd2d419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d9600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d4103310 0x7fd2d419b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d9600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4108950 0x7fd2d419fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d9600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4112bb0 con 0x7fd2d4108950 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d9600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4112a30 con 0x7fd2d4102950 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d9600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4112d30 con 0x7fd2d4103310 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 0x7fd2d419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d8c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4108950 0x7fd2d419fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 0x7fd2d419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:44810/0 (socket says 172.21.15.37:44810) 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d4103310 0x7fd2d419b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.769+0000 7fd2d3e00700 1 -- 172.21.15.37:0/1840054085 learned_addr learned my addr 172.21.15.37:0/1840054085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:55.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d3e00700 1 -- 172.21.15.37:0/1840054085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d4103310 msgr2=0x7fd2d419b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d4103310 0x7fd2d419b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d3e00700 1 -- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4108950 msgr2=0x7fd2d419fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4108950 0x7fd2d419fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d3e00700 1 -- 172.21.15.37:0/1840054085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d41a0280 con 0x7fd2d4102950 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4108950 0x7fd2d419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 0x7fd2d419b270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400ec10 tx=0x7fd2c400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d0c00700 1 -- 172.21.15.37:0/1840054085 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400cc90 con 0x7fd2d4102950 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d41a0570 con 0x7fd2d4102950 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d0c00700 1 -- 172.21.15.37:0/1840054085 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c400ce30 con 0x7fd2d4102950 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.770+0000 7fd2d0c00700 1 -- 172.21.15.37:0/1840054085 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4010430 con 0x7fd2d4102950 2024-04-26T02:01:55.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.771+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d41a7df0 con 0x7fd2d4102950 2024-04-26T02:01:55.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.774+0000 7fd2d0c00700 1 -- 172.21.15.37:0/1840054085 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2c4004000 con 0x7fd2d4102950 2024-04-26T02:01:55.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.776+0000 7fd2e5645b80 1 Processor -- start 2024-04-26T02:01:55.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.776+0000 7fd2d0c00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd2b80a3db0 0x7fd2b80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 -- start start 2024-04-26T02:01:55.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2d0c00700 1 -- 172.21.15.37:0/1840054085 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fd2c40d0a30 con 0x7fd2d4102950 2024-04-26T02:01:55.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd2b80a3db0 0x7fd2b80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616eca8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 0x55616ecb3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55616ebfbf90 con 0x55616eca9410 2024-04-26T02:01:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55616eba2830 con 0x55616ecb42c0 2024-04-26T02:01:55.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55616ebf7f50 con 0x55616eca8af0 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 0x55616ecb3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 0x55616ecb3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50320/0 (socket says 172.21.15.37:50320) 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2d3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616eca8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.777+0000 7fd2d3400700 1 -- 172.21.15.37:0/295714410 learned_addr learned my addr 172.21.15.37:0/295714410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd2b80a3db0 0x7fd2b80a6270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc003e80 tx=0x7fd2cc00b040 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d3400700 1 -- 172.21.15.37:0/295714410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 msgr2=0x55616eca8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d3400700 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616eca8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d3400700 1 -- 172.21.15.37:0/295714410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 msgr2=0x55616ecb66b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d3400700 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb66b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d3400700 1 -- 172.21.15.37:0/295714410 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616eb98780 con 0x55616eca9410 2024-04-26T02:01:55.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:55.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.778+0000 7fd2d3400700 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 0x55616ecb3d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098f0 tx=0x7fd2c000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789b9c6ffe0a4c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.779+0000 7fd2c9600700 1 -- 172.21.15.37:0/295714410 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0026070 con 0x55616eca9410 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.779+0000 7fd2c9600700 1 -- 172.21.15.37:0/295714410 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c000f910 con 0x55616eca9410 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.779+0000 7fd2c9600700 1 -- 172.21.15.37:0/295714410 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0003680 con 0x55616eca9410 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 -- 172.21.15.37:0/295714410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 msgr2=0x55616ecb3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 0x55616ecb3d80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00098f0 tx=0x7fd2c000bf60 comp rx=0 tx=0).stop 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 -- 172.21.15.37:0/295714410 shutdown_connections 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb66b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616eca8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/295714410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca9410 0x55616ecb3d80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 -- 172.21.15.37:0/295714410 >> 172.21.15.37:0/295714410 conn(0x55616ebd76a0 msgr2=0x55616eca4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 -- 172.21.15.37:0/295714410 shutdown_connections 2024-04-26T02:01:55.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 -- 172.21.15.37:0/295714410 wait complete. 2024-04-26T02:01:55.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.780+0000 7fd2e5645b80 1 -- start start 2024-04-26T02:01:55.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 Processor -- start 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecb3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55616ecb6de0 con 0x55616eca8af0 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55616ebb5960 con 0x55616ecb42c0 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55616ecb6f60 con 0x55616eca9410 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2e5645b80 10 client.0 _ll_register_callbacks cb 0x55616eb94820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:55.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.781+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44738/0 (socket says 172.21.15.37:44738) 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecb3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 -- 172.21.15.37:0/457704093 learned_addr learned my addr 172.21.15.37:0/457704093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 -- 172.21.15.37:0/457704093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 msgr2=0x55616ecb1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 -- 172.21.15.37:0/457704093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 msgr2=0x55616ecb3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecb3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 -- 172.21.15.37:0/457704093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616eb499d0 con 0x55616eca9410 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecb3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2d3400700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb0c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00053c0 tx=0x7fd2c0005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47e33a1b8e6c2ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.782+0000 7fd2be000700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:01:55.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.783+0000 7fd2be000700 1 -- 172.21.15.37:0/457704093 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0026040 con 0x55616eca9410 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.783+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c000f910 con 0x55616eca9410 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.783+0000 7fd2be000700 1 -- 172.21.15.37:0/457704093 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fd2c000f460 con 0x55616eca9410 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.783+0000 7fd2be000700 1 -- 172.21.15.37:0/457704093 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd2c00056a0 con 0x55616eca9410 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.783+0000 7fd2e5645b80 10 client.25561 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.783+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55616eb92bf0 con 0x55616eca9410 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2be000700 1 -- 172.21.15.37:0/457704093 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c000f910 con 0x55616eca9410 2024-04-26T02:01:55.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2be000700 1 -- 172.21.15.37:0/457704093 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fd2c0029070 con 0x55616eca9410 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2e5645b80 10 client.25561 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2e5645b80 10 client.25561 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2e5645b80 20 client.25561 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ecc3890 0x55616ecc5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2e5645b80 4 client.25561 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.784+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55616ec09780 con 0x55616ecc3890 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.785+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ecc3890 0x55616ecc5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.785+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ecc3890 0x55616ecc5c80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=998bacf0dbe1b9b5 server_cookie=cc01dfb7ee0934f0 in_seq=0 out_seq=0 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.785+0000 7fd2be000700 10 client.25561 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2be000700 1 -- 172.21.15.37:0/457704093 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55616ec09780 con 0x55616ecc3890 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2be000700 10 client.25561 handle_command_reply: tid=0 2024-04-26T02:01:55.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 client.25561 shutdown 2024-04-26T02:01:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 msgr2=0x55616ecb0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb0c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd2c00053c0 tx=0x7fd2c0005280 comp rx=0 tx=0).stop 2024-04-26T02:01:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 shutdown_connections 2024-04-26T02:01:55.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ecc3890 0x55616ecc5c80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecb3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecb1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/457704093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb0c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2be000700 0 client.25561 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.786+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 >> 172.21.15.37:0/457704093 conn(0x55616ebd76a0 msgr2=0x55616ecaaae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.787+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 shutdown_connections 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.787+0000 7fd2e5645b80 1 -- 172.21.15.37:0/457704093 wait complete. 2024-04-26T02:01:55.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.787+0000 7fd2e5645b80 20 client.25561 trim_cache size 0 max 16384 2024-04-26T02:01:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.808+0000 7fd2e5645b80 1 Processor -- start 2024-04-26T02:01:55.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.808+0000 7fd2e5645b80 1 -- start start 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecac480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecadee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41ad1c0 con 0x55616eca8af0 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41afa40 con 0x55616ecb42c0 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41ad830 con 0x55616eca9410 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44752/0 (socket says 172.21.15.37:44752) 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2d3400700 1 -- 172.21.15.37:0/1848559416 learned_addr learned my addr 172.21.15.37:0/1848559416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.809+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecadee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d3400700 1 -- 172.21.15.37:0/1848559416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 msgr2=0x55616ecadee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d3400700 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecadee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d3400700 1 -- 172.21.15.37:0/1848559416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 msgr2=0x55616ecac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d3400700 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d3400700 1 -- 172.21.15.37:0/1848559416 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55616ecc01f0 con 0x55616eca9410 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d8c00700 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecadee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2d3400700 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb2b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000c010 tx=0x7fd2c0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f711447d411064e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2c8c00700 1 -- 172.21.15.37:0/1848559416 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0026040 con 0x55616eca9410 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2c8c00700 1 -- 172.21.15.37:0/1848559416 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c000f910 con 0x55616eca9410 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.810+0000 7fd2c8c00700 1 -- 172.21.15.37:0/1848559416 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0003b50 con 0x55616eca9410 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1848559416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 msgr2=0x55616ecb2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb2b80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd2c000c010 tx=0x7fd2c0009c00 comp rx=0 tx=0).stop 2024-04-26T02:01:55.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1848559416 shutdown_connections 2024-04-26T02:01:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616eca8af0 0x55616ecac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616ecb42c0 0x55616ecadee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1848559416 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca9410 0x55616ecb2b80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1848559416 >> 172.21.15.37:0/1848559416 conn(0x55616ebd76a0 msgr2=0x55616ecb2470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1848559416 shutdown_connections 2024-04-26T02:01:55.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.811+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1848559416 wait complete. 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 -- start start 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 Processor -- start 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616ecc9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616eca9410 0x55616ecc7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616ecb42c0 0x55616ecc7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41adf00 con 0x55616ecb42c0 2024-04-26T02:01:55.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41af370 con 0x55616eca9410 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.812+0000 7fd2e5645b80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41ae5d0 con 0x55616eca8af0 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2e5645b80 10 client.0 _ll_register_callbacks cb 0x55616eb94820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d8c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616ecb42c0 0x55616ecc7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616eca9410 0x55616ecc7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d8c00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616ecb42c0 0x55616ecc7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50356/0 (socket says 172.21.15.37:50356) 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d8c00700 1 -- 172.21.15.37:0/1743077952 learned_addr learned my addr 172.21.15.37:0/1743077952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616ecc9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3400700 1 -- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 msgr2=0x55616ecc9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3400700 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616ecc9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3400700 1 -- 172.21.15.37:0/1743077952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616ecb42c0 msgr2=0x55616ecc7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3400700 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616ecb42c0 0x55616ecc7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.813+0000 7fd2d3400700 1 -- 172.21.15.37:0/1743077952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55616ecc8160 con 0x55616eca9410 2024-04-26T02:01:55.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2d3400700 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616eca9410 0x55616ecc7410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55616ecb23a0 tx=0x7fd2c0005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46b225c318d3027b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2be000700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2be000700 1 -- 172.21.15.37:0/1743077952 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c0026040 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2be000700 1 -- 172.21.15.37:0/1743077952 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fd2c0003680 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c000f910 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2be000700 1 -- 172.21.15.37:0/1743077952 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fd2c004d050 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2e5645b80 10 client.25499 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55616ed2a9f0 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.814+0000 7fd2be000700 1 -- 172.21.15.37:0/1743077952 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c000f910 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2be000700 1 -- 172.21.15.37:0/1743077952 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fd2c003a020 con 0x55616eca9410 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2e5645b80 10 client.25499 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2e5645b80 10 client.25499 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2e5645b80 20 client.25499 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ed2ae30 0x55616ed2d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:55.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2e5645b80 4 client.25499 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "118080ef", "release": true, "await": true}] 2024-04-26T02:01:55.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "set_id": "118080ef", "release": true, "await": true}) -- 0x55616ec03820 con 0x55616ed2ae30 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.815+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ed2ae30 0x55616ed2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2d3e00700 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ed2ae30 0x55616ed2d250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3c11ad1068e1d1c server_cookie=da2ac9277550cc31 in_seq=0 out_seq=0 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2be000700 10 client.25499 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2be000700 1 -- 172.21.15.37:0/1743077952 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55616ec03820 con 0x55616ed2ae30 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2be000700 10 client.25499 handle_command_reply: tid=0 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 client.25499 shutdown 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616eca9410 msgr2=0x55616ecc7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616eca9410 0x55616ecc7410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55616ecb23a0 tx=0x7fd2c0005e50 comp rx=0 tx=0).stop 2024-04-26T02:01:55.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 shutdown_connections 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55616ed2ae30 0x55616ed2d250 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55616ecb42c0 0x55616ecc7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2be000700 0 client.25499 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55616eca8af0 0x55616ecc9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 --2- 172.21.15.37:0/1743077952 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55616eca9410 0x55616ecc7410 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.816+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 >> 172.21.15.37:0/1743077952 conn(0x55616ebd76a0 msgr2=0x55616ecbfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.817+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 shutdown_connections 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.817+0000 7fd2e5645b80 1 -- 172.21.15.37:0/1743077952 wait complete. 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.817+0000 7fd2e5645b80 20 client.25499 trim_cache size 0 max 16384 2024-04-26T02:01:55.818 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "118080ef": { 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 12, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 60.0, 2024-04-26T02:01:55.819 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 90.0, 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:01:55.820 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd2b80a3db0 msgr2=0x7fd2b80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd2b80a3db0 0x7fd2b80a6270 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc003e80 tx=0x7fd2cc00b040 comp rx=0 tx=0).stop 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 msgr2=0x7fd2d419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 0x7fd2d419b270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400ec10 tx=0x7fd2c400ef20 comp rx=0 tx=0).stop 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 shutdown_connections 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2d4108950 0x7fd2d419fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fd2b80a3db0 0x7fd2b80a6270 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fd2d4102950 0x7fd2d419b270 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 --2- 172.21.15.37:0/1840054085 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fd2d4103310 0x7fd2d419b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 >> 172.21.15.37:0/1840054085 conn(0x7fd2d40fe460 msgr2=0x7fd2d410e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 shutdown_connections 2024-04-26T02:01:55.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:01:55.818+0000 7fd2d9600700 1 -- 172.21.15.37:0/1840054085 wait complete. 2024-04-26T02:01:56.392 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:56 smithi037 ceph-mon[27906]: pgmap v122: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:56 smithi071 ceph-mon[32578]: pgmap v122: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:56 smithi045 ceph-mon[32520]: pgmap v122: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:56.878 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '118080ef', seconds elapsed: 47.2 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 -- 172.21.15.37:0/3644037257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab48104d70 msgr2=0x7fab48108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 --2- 172.21.15.37:0/3644037257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab48104d70 0x7fab48108db0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fab30009900 tx=0x7fab3002a3f0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 -- 172.21.15.37:0/3644037257 shutdown_connections 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 --2- 172.21.15.37:0/3644037257 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab48109490 0x7fab48111060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 --2- 172.21.15.37:0/3644037257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab48104d70 0x7fab48108db0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 --2- 172.21.15.37:0/3644037257 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab481043c0 0x7fab481047a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 -- 172.21.15.37:0/3644037257 >> 172.21.15.37:0/3644037257 conn(0x7fab480ffc40 msgr2=0x7fab48102060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 -- 172.21.15.37:0/3644037257 shutdown_connections 2024-04-26T02:01:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.461+0000 7fab4d200640 1 -- 172.21.15.37:0/3644037257 wait complete. 2024-04-26T02:01:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.463+0000 7fab4d200640 1 Processor -- start 2024-04-26T02:01:57.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.463+0000 7fab4d200640 1 -- start start 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab4d200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab481043c0 0x7fab481a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab4d200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab48109490 0x7fab481a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab4d200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab4819a770 0x7fab4819abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab4d200640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab481143d0 con 0x7fab4819a770 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab4d200640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab48114250 con 0x7fab48109490 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab4d200640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab48114550 con 0x7fab481043c0 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab4819a770 0x7fab4819abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab4819a770 0x7fab4819abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50386/0 (socket says 172.21.15.37:50386) 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab46800640 1 -- 172.21.15.37:0/2432150154 learned_addr learned my addr 172.21.15.37:0/2432150154 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab45e00640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab481043c0 0x7fab481a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.464+0000 7fab3d400640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab48109490 0x7fab481a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab45e00640 1 -- 172.21.15.37:0/2432150154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab48109490 msgr2=0x7fab481a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab45e00640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab48109490 0x7fab481a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab45e00640 1 -- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab4819a770 msgr2=0x7fab4819abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab45e00640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab4819a770 0x7fab4819abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab45e00640 1 -- 172.21.15.37:0/2432150154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab4819b370 con 0x7fab481043c0 2024-04-26T02:01:57.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab45e00640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab481043c0 0x7fab481a06b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab3800eb00 tx=0x7fab3800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab3f400640 1 -- 172.21.15.37:0/2432150154 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38010430 con 0x7fab481043c0 2024-04-26T02:01:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab481a7450 con 0x7fab481043c0 2024-04-26T02:01:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab3f400640 1 -- 172.21.15.37:0/2432150154 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab38010a80 con 0x7fab481043c0 2024-04-26T02:01:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab3f400640 1 -- 172.21.15.37:0/2432150154 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab38017a40 con 0x7fab481043c0 2024-04-26T02:01:57.465 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.465+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab481a7850 con 0x7fab481043c0 2024-04-26T02:01:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.470+0000 7fab5005c740 1 Processor -- start 2024-04-26T02:01:57.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.470+0000 7fab5005c740 1 -- start start 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab5005c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5af9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab5005c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b044c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab3f400640 1 -- 172.21.15.37:0/2432150154 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab38017c60 con 0x7fab481043c0 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab5005c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b06df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab5005c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616f5a5f900 con 0x5616f5b04a00 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab5005c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5616f5a05ef0 con 0x5616f5af9820 2024-04-26T02:01:57.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab5005c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5616f5a5b8c0 con 0x5616f5afa140 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab46800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5af9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab46800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5af9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:44900/0 (socket says 172.21.15.37:44900) 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab46800640 1 -- 172.21.15.37:0/2344605156 learned_addr learned my addr 172.21.15.37:0/2344605156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab45e00640 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b044c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab3d400640 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b06df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab46800640 1 -- 172.21.15.37:0/2344605156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 msgr2=0x5616f5b044c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.471+0000 7fab46800640 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b044c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab46800640 1 -- 172.21.15.37:0/2344605156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 msgr2=0x5616f5b06df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab46800640 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b06df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab46800640 1 -- 172.21.15.37:0/2344605156 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f5772210 con 0x5616f5af9820 2024-04-26T02:01:57.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab46800640 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5af9c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d20 tx=0x7fab40011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f30fb3ea4b9eccbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.474 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab37400640 1 -- 172.21.15.37:0/2344605156 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4001e070 con 0x5616f5af9820 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab3f400640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab2c0a3d10 0x7fab2c0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab37400640 1 -- 172.21.15.37:0/2344605156 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab40019ae0 con 0x5616f5af9820 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.472+0000 7fab37400640 1 -- 172.21.15.37:0/2344605156 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4000fa20 con 0x5616f5af9820 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab3f400640 1 -- 172.21.15.37:0/2432150154 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab380d1250 con 0x7fab481043c0 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab45e00640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab2c0a3d10 0x7fab2c0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab5005c740 1 -- 172.21.15.37:0/2344605156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 msgr2=0x5616f5af9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab45e00640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab2c0a3d10 0x7fab2c0a61d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5616f5afadc0 tx=0x7fab380d4d00 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab5005c740 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5af9c00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fab40009d20 tx=0x7fab40011810 comp rx=0 tx=0).stop 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab5005c740 1 -- 172.21.15.37:0/2344605156 shutdown_connections 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.473+0000 7fab5005c740 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b06df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 --2- 172.21.15.37:0/2344605156 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5af9c00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.475 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 -- 172.21.15.37:0/2344605156 >> 172.21.15.37:0/2344605156 conn(0x5616f5af5d30 msgr2=0x5616f5af6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 -- 172.21.15.37:0/2344605156 shutdown_connections 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 -- 172.21.15.37:0/2344605156 wait complete. 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 -- start start 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.474+0000 7fab5005c740 1 Processor -- start 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b01e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616f5b07630 con 0x5616f5b04a00 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5616f5a19310 con 0x5616f5af9820 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5616f5b077b0 con 0x5616f5afa140 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab5005c740 10 client.0 _ll_register_callbacks cb 0x5616f59e7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab3d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b01e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab3d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b01e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44786/0 (socket says 172.21.15.37:44786) 2024-04-26T02:01:57.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab3d400640 1 -- 172.21.15.37:0/2164357713 learned_addr learned my addr 172.21.15.37:0/2164357713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.475+0000 7fab46800640 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab3d400640 1 -- 172.21.15.37:0/2164357713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 msgr2=0x5616f5b01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab3d400640 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab3d400640 1 -- 172.21.15.37:0/2164357713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 msgr2=0x5616f5b023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab3d400640 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab3d400640 1 -- 172.21.15.37:0/2164357713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f583e620 con 0x5616f5afa140 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab3d400640 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b01e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab30009900 tx=0x7fab30032de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecbf5e31dc88956f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab2be00640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab2be00640 1 -- 172.21.15.37:0/2164357713 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30035040 con 0x5616f5afa140 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab2be00640 1 -- 172.21.15.37:0/2164357713 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab30031760 con 0x5616f5afa140 2024-04-26T02:01:57.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.476+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab40019ae0 con 0x5616f5afa140 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab2be00640 1 -- 172.21.15.37:0/2164357713 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fab30004040 con 0x5616f5afa140 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab5005c740 10 client.25579 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5616f5b13830 con 0x5616f5afa140 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab2be00640 1 -- 172.21.15.37:0/2164357713 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab40019ae0 con 0x5616f5afa140 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab2be00640 1 -- 172.21.15.37:0/2164357713 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fab30004750 con 0x5616f5afa140 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab5005c740 10 client.25579 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab5005c740 10 client.25579 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.477+0000 7fab5005c740 20 client.25579 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.478+0000 7fab5005c740 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b14cf0 0x5616f5b170e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.478+0000 7fab5005c740 4 client.25579 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.478+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616f5a738b0 con 0x5616f5b14cf0 2024-04-26T02:01:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.478+0000 7fab46800640 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b14cf0 0x5616f5b170e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.478+0000 7fab46800640 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b14cf0 0x5616f5b170e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0576706b2fedd73 server_cookie=9117135d3f96d1e6 in_seq=0 out_seq=0 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.478+0000 7fab2be00640 10 client.25579 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.479+0000 7fab2be00640 1 -- 172.21.15.37:0/2164357713 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5616f5a738b0 con 0x5616f5b14cf0 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.479+0000 7fab2be00640 10 client.25579 handle_command_reply: tid=0 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.479+0000 7fab5005c740 1 client.25579 shutdown 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 msgr2=0x5616f5b01e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b01e70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fab30009900 tx=0x7fab30032de0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 shutdown_connections 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b14cf0 0x5616f5b170e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5b04a00 0x5616f5b023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b01e70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab2be00640 0 client.25579 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 --2- 172.21.15.37:0/2164357713 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 >> 172.21.15.37:0/2164357713 conn(0x5616f5af5d30 msgr2=0x5616f5afba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 shutdown_connections 2024-04-26T02:01:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 1 -- 172.21.15.37:0/2164357713 wait complete. 2024-04-26T02:01:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.480+0000 7fab5005c740 20 client.25579 trim_cache size 0 max 16384 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.496+0000 7fab5005c740 1 Processor -- start 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.496+0000 7fab5005c740 1 -- start start 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab5005c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab5005c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 0x5616f5af5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab5005c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5b04a00 0x5616f5afb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab5005c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab481adce0 con 0x5616f5afa140 2024-04-26T02:01:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab5005c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab481b0560 con 0x5616f5af9820 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab5005c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab481ae350 con 0x5616f5b04a00 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab3d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 0x5616f5af5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab3d400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 0x5616f5af5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50420/0 (socket says 172.21.15.37:50420) 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab3d400640 1 -- 172.21.15.37:0/3886632501 learned_addr learned my addr 172.21.15.37:0/3886632501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab46800640 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5b04a00 0x5616f5afb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.497+0000 7fab46800640 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab3d400640 1 -- 172.21.15.37:0/3886632501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5b04a00 msgr2=0x5616f5afb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab3d400640 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5b04a00 0x5616f5afb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab3d400640 1 -- 172.21.15.37:0/3886632501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 msgr2=0x5616f5b104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab3d400640 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab3d400640 1 -- 172.21.15.37:0/3886632501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f5b17740 con 0x5616f5afa140 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab46800640 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5b04a00 0x5616f5afb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:01:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab3d400640 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 0x5616f5af5610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab30000c00 tx=0x7fab3002aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e0689792646d78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab36a00640 1 -- 172.21.15.37:0/3886632501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30035030 con 0x5616f5afa140 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab36a00640 1 -- 172.21.15.37:0/3886632501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab300046c0 con 0x5616f5afa140 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.498+0000 7fab36a00640 1 -- 172.21.15.37:0/3886632501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab300038d0 con 0x5616f5afa140 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 -- 172.21.15.37:0/3886632501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 msgr2=0x5616f5af5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 0x5616f5af5610 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fab30000c00 tx=0x7fab3002aaf0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 -- 172.21.15.37:0/3886632501 shutdown_connections 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5b04a00 0x5616f5afb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5afa140 0x5616f5af5610 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 --2- 172.21.15.37:0/3886632501 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5af9820 0x5616f5b104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.500 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 -- 172.21.15.37:0/3886632501 >> 172.21.15.37:0/3886632501 conn(0x5616f5af5d30 msgr2=0x5616f5afb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 -- 172.21.15.37:0/3886632501 shutdown_connections 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.499+0000 7fab5005c740 1 -- 172.21.15.37:0/3886632501 wait complete. 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 -- start start 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 Processor -- start 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 0x5616f5b0cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b0a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5b04a00 0x5616f5b0aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab481aea20 con 0x5616f5af9820 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab481afe90 con 0x5616f5b04a00 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.500+0000 7fab5005c740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab481af0f0 con 0x5616f5afa140 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab5005c740 10 client.0 _ll_register_callbacks cb 0x5616f59e7cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 0x5616f5b0cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab3d400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b0a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 0x5616f5b0cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50434/0 (socket says 172.21.15.37:50434) 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 -- 172.21.15.37:0/224474188 learned_addr learned my addr 172.21.15.37:0/224474188 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5b04a00 0x5616f5b0aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 -- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 msgr2=0x5616f5b0a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b0a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 -- 172.21.15.37:0/224474188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5b04a00 msgr2=0x5616f5b0aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5b04a00 0x5616f5b0aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab46800640 1 -- 172.21.15.37:0/224474188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f5b7a230 con 0x5616f5af9820 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.501+0000 7fab3d400640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab46800640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 0x5616f5b0cf20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab4000f6b0 tx=0x7fab40011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1b00e59d47c72b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:01:57.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab2be00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:01:57.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab2be00640 1 -- 172.21.15.37:0/224474188 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab4001e040 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab2be00640 1 -- 172.21.15.37:0/224474188 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab40019700 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab300046c0 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab2be00640 1 -- 172.21.15.37:0/224474188 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fab40019900 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab5005c740 10 client.15435 fetch_fsmap learned FSMap version 19 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.502+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5616f5b7b3a0 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab2be00640 1 -- 172.21.15.37:0/224474188 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab300046c0 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab2be00640 1 -- 172.21.15.37:0/224474188 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fab4000fae0 con 0x5616f5af9820 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab5005c740 10 client.15435 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab5005c740 10 client.15435 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab5005c740 20 client.15435 populate_metadata read hostname 'smithi037' 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab5005c740 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b7b8d0 0x5616f5b7dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab5005c740 4 client.15435 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5616f5a6d320 con 0x5616f5b7b8d0 2024-04-26T02:01:57.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.503+0000 7fab3d400640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b7b8d0 0x5616f5b7dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:01:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.504+0000 7fab3d400640 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b7b8d0 0x5616f5b7dcc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2279bffc44701981 server_cookie=e58d1aa60f706395 in_seq=0 out_seq=0 2024-04-26T02:01:57.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.504+0000 7fab2be00640 10 client.15435 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab2be00640 1 -- 172.21.15.37:0/224474188 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5616f5a6d320 con 0x5616f5b7b8d0 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab2be00640 10 client.15435 handle_command_reply: tid=0 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 client.15435 shutdown 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 msgr2=0x5616f5b0cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 0x5616f5b0cf20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab4000f6b0 tx=0x7fab40011810 comp rx=0 tx=0).stop 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 shutdown_connections 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5616f5b7b8d0 0x5616f5b7dcc0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5616f5b04a00 0x5616f5b0aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5616f5afa140 0x5616f5b0a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 --2- 172.21.15.37:0/224474188 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f5af9820 0x5616f5b0cf20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 >> 172.21.15.37:0/224474188 conn(0x5616f5af5d30 msgr2=0x5616f5b018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.508+0000 7fab2be00640 0 client.15435 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 shutdown_connections 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab5005c740 1 -- 172.21.15.37:0/224474188 wait complete. 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab5005c740 20 client.15435 trim_cache size 0 max 16384 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:01:57.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:01:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab2c0a3d10 msgr2=0x7fab2c0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab4d200640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab2c0a3d10 0x7fab2c0a61d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x5616f5afadc0 tx=0x7fab380d4d00 comp rx=0 tx=0).stop 2024-04-26T02:01:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab481043c0 msgr2=0x7fab481a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:01:57.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.509+0000 7fab4d200640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab481043c0 0x7fab481a06b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab3800eb00 tx=0x7fab3800c2d0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 shutdown_connections 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab2c0a3d10 0x7fab2c0a61d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab4819a770 0x7fab4819abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab48109490 0x7fab481a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 --2- 172.21.15.37:0/2432150154 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab481043c0 0x7fab481a06b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 >> 172.21.15.37:0/2432150154 conn(0x7fab480ffc40 msgr2=0x7fab48101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 shutdown_connections 2024-04-26T02:01:57.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:01:57.510+0000 7fab4d200640 1 -- 172.21.15.37:0/2432150154 wait complete. 2024-04-26T02:01:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:01:58 smithi071 ceph-mon[32578]: pgmap v123: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:01:58 smithi037 ceph-mon[27906]: pgmap v123: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:01:58.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:01:58 smithi045 ceph-mon[32520]: pgmap v123: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1.2 KiB/s wr, 3 op/s 2024-04-26T02:02:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:00 smithi071 ceph-mon[32578]: pgmap v124: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-26T02:02:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:00 smithi037 ceph-mon[27906]: pgmap v124: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-26T02:02:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:00 smithi045 ceph-mon[32520]: pgmap v124: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-04-26T02:02:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:02 smithi071 ceph-mon[32578]: pgmap v125: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T02:02:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:02 smithi037 ceph-mon[27906]: pgmap v125: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T02:02:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:02 smithi045 ceph-mon[32520]: pgmap v125: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 255 B/s wr, 1 op/s 2024-04-26T02:02:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:04 smithi037 ceph-mon[27906]: pgmap v126: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 103 MiB/s wr, 44 op/s 2024-04-26T02:02:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:04 smithi045 ceph-mon[32520]: pgmap v126: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 103 MiB/s wr, 44 op/s 2024-04-26T02:02:04.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:04 smithi071 ceph-mon[32578]: pgmap v126: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 103 MiB/s wr, 44 op/s 2024-04-26T02:02:06.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:06 smithi037 ceph-mon[27906]: pgmap v127: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 103 MiB/s wr, 43 op/s 2024-04-26T02:02:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:06 smithi045 ceph-mon[32520]: pgmap v127: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 103 MiB/s wr, 43 op/s 2024-04-26T02:02:06.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:06 smithi071 ceph-mon[32578]: pgmap v127: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 103 MiB/s wr, 43 op/s 2024-04-26T02:02:08.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:08 smithi045 ceph-mon[32520]: pgmap v128: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 103 MiB/s wr, 43 op/s 2024-04-26T02:02:08.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:08 smithi071 ceph-mon[32578]: pgmap v128: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 103 MiB/s wr, 43 op/s 2024-04-26T02:02:08.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:08 smithi037 ceph-mon[27906]: pgmap v128: 97 pgs: 97 active+clean; 12 GiB data, 36 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 103 MiB/s wr, 43 op/s 2024-04-26T02:02:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:10 smithi045 ceph-mon[32520]: pgmap v129: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 252 MiB/s wr, 82 op/s 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 -- 172.21.15.37:0/2282186445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 msgr2=0x7f311c0746e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 --2- 172.21.15.37:0/2282186445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c0746e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f310c009900 tx=0x7f310c02a480 comp rx=0 tx=0).stop 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 -- 172.21.15.37:0/2282186445 shutdown_connections 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 --2- 172.21.15.37:0/2282186445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311c10e800 0x7f311c110bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 --2- 172.21.15.37:0/2282186445 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c074c20 0x7f311c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 --2- 172.21.15.37:0/2282186445 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c0746e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 -- 172.21.15.37:0/2282186445 >> 172.21.15.37:0/2282186445 conn(0x7f311c0fda10 msgr2=0x7f311c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 -- 172.21.15.37:0/2282186445 shutdown_connections 2024-04-26T02:02:10.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.638+0000 7f3122800640 1 -- 172.21.15.37:0/2282186445 wait complete. 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.639+0000 7f3122800640 1 Processor -- start 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.639+0000 7f3122800640 1 -- start start 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f3122800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c1a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f3122800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311c074c20 0x7f311c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f3122800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c10e800 0x7f311c19a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f3122800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1143b0 con 0x7f311c074c20 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f3122800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f311c114230 con 0x7f311c074300 2024-04-26T02:02:10.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f3122800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f311c114530 con 0x7f311c10e800 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c1a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c1a06b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:36992/0 (socket says 172.21.15.37:36992) 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 -- 172.21.15.37:0/2369550963 learned_addr learned my addr 172.21.15.37:0/2369550963 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311be00640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c10e800 0x7f311c19a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311aa00640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311c074c20 0x7f311c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 -- 172.21.15.37:0/2369550963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c10e800 msgr2=0x7f311c19a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c10e800 0x7f311c19a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 -- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311c074c20 msgr2=0x7f311c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311c074c20 0x7f311c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311b400640 1 -- 172.21.15.37:0/2369550963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f311c19b030 con 0x7f311c074300 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.640+0000 7f311be00640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c10e800 0x7f311c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:10.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.641+0000 7f311b400640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c1a06b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f310c005940 tx=0x7f310c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.641+0000 7f3113e00640 1 -- 172.21.15.37:0/2369550963 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c03f030 con 0x7f311c074300 2024-04-26T02:02:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.641+0000 7f3113e00640 1 -- 172.21.15.37:0/2369550963 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f310c0044c0 con 0x7f311c074300 2024-04-26T02:02:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.641+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f311c19b2c0 con 0x7f311c074300 2024-04-26T02:02:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.641+0000 7f3113e00640 1 -- 172.21.15.37:0/2369550963 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c035070 con 0x7f311c074300 2024-04-26T02:02:10.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.641+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f311c1a7450 con 0x7f311c074300 2024-04-26T02:02:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.644+0000 7f3125659740 1 Processor -- start 2024-04-26T02:02:10.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 -- start start 2024-04-26T02:02:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dadc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dae140 0x55b9d1db84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1dbadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1d13900 con 0x55b9d1dae140 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1cb9ef0 con 0x55b9d1dad820 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f3125659740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1d0f8c0 con 0x55b9d1db8a00 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dadc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dadc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:36996/0 (socket says 172.21.15.37:36996) 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 -- 172.21.15.37:0/3596413145 learned_addr learned my addr 172.21.15.37:0/3596413145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311aa00640 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1dbadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 -- 172.21.15.37:0/3596413145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 msgr2=0x55b9d1dbadf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1dbadf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 -- 172.21.15.37:0/3596413145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dae140 msgr2=0x55b9d1db84c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.645+0000 7f311be00640 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dae140 0x55b9d1db84c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.646+0000 7f311be00640 1 -- 172.21.15.37:0/3596413145 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d1a26210 con 0x55b9d1dad820 2024-04-26T02:02:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.646+0000 7f311be00640 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dadc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3114011790 tx=0x7f3114011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=666cdf8a51dded32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.646+0000 7f3113e00640 1 -- 172.21.15.37:0/2369550963 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f310c042030 con 0x7f311c074300 2024-04-26T02:02:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.646+0000 7f3110c00640 1 -- 172.21.15.37:0/3596413145 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3114003c30 con 0x55b9d1dad820 2024-04-26T02:02:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.646+0000 7f3110c00640 1 -- 172.21.15.37:0/3596413145 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311400fe50 con 0x55b9d1dad820 2024-04-26T02:02:10.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.646+0000 7f3110c00640 1 -- 172.21.15.37:0/3596413145 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31140193e0 con 0x55b9d1dad820 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 -- 172.21.15.37:0/3596413145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 msgr2=0x55b9d1dadc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dadc00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f3114011790 tx=0x7f3114011c50 comp rx=0 tx=0).stop 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 -- 172.21.15.37:0/3596413145 shutdown_connections 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1dbadf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dae140 0x55b9d1db84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 --2- 172.21.15.37:0/3596413145 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dadc00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 -- 172.21.15.37:0/3596413145 >> 172.21.15.37:0/3596413145 conn(0x55b9d1da9d30 msgr2=0x55b9d1daa110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3113e00640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f31000a3d10 0x7f31000a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.647+0000 7f3125659740 1 -- 172.21.15.37:0/3596413145 shutdown_connections 2024-04-26T02:02:10.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.648+0000 7f3113e00640 1 -- 172.21.15.37:0/2369550963 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f310c0f0be0 con 0x7f311c074300 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.648+0000 7f3125659740 1 -- 172.21.15.37:0/3596413145 wait complete. 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.648+0000 7f311aa00640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f31000a3d10 0x7f31000a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.648+0000 7f3125659740 1 -- start start 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.648+0000 7f3125659740 1 Processor -- start 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f311aa00640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f31000a3d10 0x7f31000a61d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b9d1dab150 tx=0x7f3108005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dad820 0x55b9d1db5960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1db5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1dbb630 con 0x55b9d1db8a00 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1ccd310 con 0x55b9d1dae140 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55b9d1dbb7b0 con 0x55b9d1dad820 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f3125659740 10 client.0 _ll_register_callbacks cb 0x55b9d1c9bcb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:10.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f311be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f311be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54160/0 (socket says 172.21.15.37:54160) 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f311be00640 1 -- 172.21.15.37:0/1553472972 learned_addr learned my addr 172.21.15.37:0/1553472972 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f311b400640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dad820 0x55b9d1db5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.649+0000 7f311aa00640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1db5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311b400640 1 -- 172.21.15.37:0/1553472972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 msgr2=0x55b9d1db5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311b400640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1db5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311b400640 1 -- 172.21.15.37:0/1553472972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 msgr2=0x55b9d1db63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311b400640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311b400640 1 -- 172.21.15.37:0/1553472972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d1af2620 con 0x55b9d1dad820 2024-04-26T02:02:10.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311aa00640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1db5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f311b400640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dad820 0x55b9d1db5960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f310c0ee300 tx=0x7f310c0bc710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92785f3c6f89dec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f3105600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f3105600640 1 -- 172.21.15.37:0/1553472972 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310c049040 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f3105600640 1 -- 172.21.15.37:0/1553472972 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f310c0b91c0 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.650+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f311400fe50 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3105600640 1 -- 172.21.15.37:0/1553472972 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f310c0bcb50 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 10 client.25597 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b9d1dbb630 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3105600640 1 -- 172.21.15.37:0/1553472972 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f311400fe50 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3105600640 1 -- 172.21.15.37:0/1553472972 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f310c0c5330 con 0x55b9d1dad820 2024-04-26T02:02:10.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 10 client.25597 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 10 client.25597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 20 client.25597 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1dc8d50 0x55b9d1dcb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 4 client.25597 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.651+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9d1d278b0 con 0x55b9d1dc8d50 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.652+0000 7f311aa00640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1dc8d50 0x55b9d1dcb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.652+0000 7f311aa00640 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1dc8d50 0x55b9d1dcb140 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d3e34dbe92aea4 server_cookie=92d7c2db5b17081e in_seq=0 out_seq=0 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.652+0000 7f3105600640 10 client.25597 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.653+0000 7f3105600640 1 -- 172.21.15.37:0/1553472972 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b9d1d278b0 con 0x55b9d1dc8d50 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.653+0000 7f3105600640 10 client.25597 handle_command_reply: tid=0 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.653+0000 7f3125659740 1 client.25597 shutdown 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dad820 msgr2=0x55b9d1db5960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dad820 0x55b9d1db5960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f310c0ee300 tx=0x7f310c0bc710 comp rx=0 tx=0).stop 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 shutdown_connections 2024-04-26T02:02:10.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1dc8d50 0x55b9d1dcb140 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1db5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3105600640 0 client.25597 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 --2- 172.21.15.37:0/1553472972 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dad820 0x55b9d1db5960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 >> 172.21.15.37:0/1553472972 conn(0x55b9d1da9d30 msgr2=0x55b9d1dafab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 shutdown_connections 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 1 -- 172.21.15.37:0/1553472972 wait complete. 2024-04-26T02:02:10.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.654+0000 7f3125659740 20 client.25597 trim_cache size 0 max 16384 2024-04-26T02:02:10.656 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:10 smithi071 ceph-mon[32578]: pgmap v129: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 252 MiB/s wr, 82 op/s 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.685+0000 7f3125659740 1 Processor -- start 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.685+0000 7f3125659740 1 -- start start 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f3125659740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dc44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f3125659740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 0x55b9d1db01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f3125659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f3125659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1add40 con 0x55b9d1db8a00 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f3125659740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1b05c0 con 0x55b9d1dad820 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f3125659740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1ae3b0 con 0x55b9d1dae140 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f311b400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dc44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f311aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 0x55b9d1db01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f311aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 0x55b9d1db01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:49192/0 (socket says 172.21.15.37:49192) 2024-04-26T02:02:10.686 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f311aa00640 1 -- 172.21.15.37:0/3781585621 learned_addr learned my addr 172.21.15.37:0/3781585621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.686+0000 7f311be00640 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.687 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.687+0000 7f311aa00640 1 -- 172.21.15.37:0/3781585621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 msgr2=0x55b9d1dc44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.687+0000 7f311aa00640 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dc44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.687+0000 7f311aa00640 1 -- 172.21.15.37:0/3781585621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 msgr2=0x55b9d1db6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.687+0000 7f311aa00640 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.687+0000 7f311aa00640 1 -- 172.21.15.37:0/3781585621 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9d1dcb770 con 0x55b9d1dae140 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.687+0000 7f311aa00640 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 0x55b9d1db01e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55b9d1db51c0 tx=0x7f3108019000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=277ed96353e92b6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.688+0000 7f3107e00640 1 -- 172.21.15.37:0/3781585621 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3108031070 con 0x55b9d1dae140 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.688+0000 7f3107e00640 1 -- 172.21.15.37:0/3781585621 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3108003d00 con 0x55b9d1dae140 2024-04-26T02:02:10.688 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.688+0000 7f3107e00640 1 -- 172.21.15.37:0/3781585621 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f310802a750 con 0x55b9d1dae140 2024-04-26T02:02:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 -- 172.21.15.37:0/3781585621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 msgr2=0x55b9d1db01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 0x55b9d1db01e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x55b9d1db51c0 tx=0x7f3108019000 comp rx=0 tx=0).stop 2024-04-26T02:02:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 -- 172.21.15.37:0/3781585621 shutdown_connections 2024-04-26T02:02:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1db8a00 0x55b9d1db6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1dae140 0x55b9d1db01e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 --2- 172.21.15.37:0/3781585621 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dad820 0x55b9d1dc44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 -- 172.21.15.37:0/3781585621 >> 172.21.15.37:0/3781585621 conn(0x55b9d1da9d30 msgr2=0x55b9d1db5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 -- 172.21.15.37:0/3781585621 shutdown_connections 2024-04-26T02:02:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.689+0000 7f3125659740 1 -- 172.21.15.37:0/3781585621 wait complete. 2024-04-26T02:02:10.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.690+0000 7f3125659740 1 -- start start 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.690+0000 7f3125659740 1 Processor -- start 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dad820 0x55b9d1dbdc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1dbe160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1e2d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1aea80 con 0x55b9d1dad820 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1afef0 con 0x55b9d1dae140 2024-04-26T02:02:10.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f311c1af150 con 0x55b9d1db8a00 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f3125659740 10 client.0 _ll_register_callbacks cb 0x7f310c0b8ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f311aa00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1dbe160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f311b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dad820 0x55b9d1dbdc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f311b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dad820 0x55b9d1dbdc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54190/0 (socket says 172.21.15.37:54190) 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f311b400640 1 -- 172.21.15.37:0/3894411299 learned_addr learned my addr 172.21.15.37:0/3894411299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.691+0000 7f311be00640 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1e2d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f311be00640 1 -- 172.21.15.37:0/3894411299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 msgr2=0x55b9d1dbe160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f311be00640 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1dbe160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f311be00640 1 -- 172.21.15.37:0/3894411299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dad820 msgr2=0x55b9d1dbdc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f311be00640 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dad820 0x55b9d1dbdc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f311be00640 1 -- 172.21.15.37:0/3894411299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9d1e2d880 con 0x55b9d1db8a00 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f311be00640 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1e2d270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f311400f790 tx=0x7f31140041b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=942b0e505cc7b84c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f3105600640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:02:10.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.692+0000 7f3105600640 1 -- 172.21.15.37:0/3894411299 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f311402a020 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3105600640 1 -- 172.21.15.37:0/3894411299 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f311400f9a0 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3108003d00 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3105600640 1 -- 172.21.15.37:0/3894411299 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f3114035050 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3125659740 10 client.25603 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55b9d1e2eb90 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3105600640 1 -- 172.21.15.37:0/3894411299 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3108003d00 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3105600640 1 -- 172.21.15.37:0/3894411299 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f3114018070 con 0x55b9d1db8a00 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3125659740 10 client.25603 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.693+0000 7f3125659740 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f3125659740 20 client.25603 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f3125659740 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1e2f190 0x55b9d1e315b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f3125659740 4 client.25603 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9d1d21320 con 0x55b9d1e2f190 2024-04-26T02:02:10.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f311b400640 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1e2f190 0x55b9d1e315b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f3105600640 10 client.25603 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:10.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.694+0000 7f311b400640 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1e2f190 0x55b9d1e315b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a53797e734e8f60 server_cookie=c2f8bf1da5b981b in_seq=0 out_seq=0 2024-04-26T02:02:10.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:10 smithi037 ceph-mon[27906]: pgmap v129: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 252 MiB/s wr, 82 op/s 2024-04-26T02:02:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.754+0000 7f3105600640 1 -- 172.21.15.37:0/3894411299 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9d1d21320 con 0x55b9d1e2f190 2024-04-26T02:02:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.754+0000 7f3105600640 10 client.25603 handle_command_reply: tid=0 2024-04-26T02:02:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.754+0000 7f3125659740 1 client.25603 shutdown 2024-04-26T02:02:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.756+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 msgr2=0x55b9d1e2d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.756+0000 7f3125659740 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1e2d270 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f311400f790 tx=0x7f31140041b0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 shutdown_connections 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55b9d1e2f190 0x55b9d1e315b0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55b9d1db8a00 0x55b9d1e2d270 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55b9d1dae140 0x55b9d1dbe160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 --2- 172.21.15.37:0/3894411299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9d1dad820 0x55b9d1dbdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 >> 172.21.15.37:0/3894411299 conn(0x55b9d1da9d30 msgr2=0x55b9d1da8e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3105600640 0 client.25603 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.758+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 shutdown_connections 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.759+0000 7f3125659740 1 -- 172.21.15.37:0/3894411299 wait complete. 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.759+0000 7f3125659740 20 client.25603 trim_cache size 0 max 16384 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:02:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:02:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.763+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f31000a3d10 msgr2=0x7f31000a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.762 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.763+0000 7f3122800640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f31000a3d10 0x7f31000a61d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x55b9d1dab150 tx=0x7f3108005f90 comp rx=0 tx=0).stop 2024-04-26T02:02:10.764 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 msgr2=0x7f311c1a06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c1a06b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f310c005940 tx=0x7f310c0040c0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 shutdown_connections 2024-04-26T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f31000a3d10 0x7f31000a61d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f311c10e800 0x7f311c19a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f311c074c20 0x7f311c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 --2- 172.21.15.37:0/2369550963 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f311c074300 0x7f311c1a06b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.764+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 >> 172.21.15.37:0/2369550963 conn(0x7f311c0fda10 msgr2=0x7f311c0ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:10.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.765+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 shutdown_connections 2024-04-26T02:02:10.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:10.768+0000 7f3122800640 1 -- 172.21.15.37:0/2369550963 wait complete. 2024-04-26T02:02:11.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:11 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:11.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:11 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:11.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:11 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:02:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:12 smithi071 ceph-mon[32578]: pgmap v130: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 252 MiB/s wr, 81 op/s 2024-04-26T02:02:12.655 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:12 smithi045 ceph-mon[32520]: pgmap v130: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 252 MiB/s wr, 81 op/s 2024-04-26T02:02:12.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:12 smithi037 ceph-mon[27906]: pgmap v130: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 252 MiB/s wr, 81 op/s 2024-04-26T02:02:13.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:13 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:02:13.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:13 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:02:13.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:13 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:02:13.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:13 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:02:13.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:13 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:02:13.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:13 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:02:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:13 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:02:13.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:13 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:02:13.755 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:13 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:02:14.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:14 smithi071 ceph-mon[32578]: pgmap v131: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 360 MiB/s wr, 113 op/s 2024-04-26T02:02:14.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:14 smithi037 ceph-mon[27906]: pgmap v131: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 360 MiB/s wr, 113 op/s 2024-04-26T02:02:14.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:14 smithi045 ceph-mon[32520]: pgmap v131: 97 pgs: 97 active+clean; 15 GiB data, 46 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 360 MiB/s wr, 113 op/s 2024-04-26T02:02:15.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:15 smithi071 ceph-mon[32578]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 288 MiB/s wr, 78 op/s 2024-04-26T02:02:15.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:15 smithi037 ceph-mon[27906]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 288 MiB/s wr, 78 op/s 2024-04-26T02:02:15.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:15 smithi045 ceph-mon[32520]: pgmap v132: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 288 MiB/s wr, 78 op/s 2024-04-26T02:02:18.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:17 smithi071 ceph-mon[32578]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 288 MiB/s wr, 78 op/s 2024-04-26T02:02:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:17 smithi037 ceph-mon[27906]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 288 MiB/s wr, 78 op/s 2024-04-26T02:02:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:17 smithi045 ceph-mon[32520]: pgmap v133: 97 pgs: 97 active+clean; 16 GiB data, 47 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 288 MiB/s wr, 78 op/s 2024-04-26T02:02:20.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:19 smithi071 ceph-mon[32578]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 30 KiB/s rd, 456 MiB/s wr, 122 op/s 2024-04-26T02:02:20.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:19 smithi037 ceph-mon[27906]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 30 KiB/s rd, 456 MiB/s wr, 122 op/s 2024-04-26T02:02:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:19 smithi045 ceph-mon[32520]: pgmap v134: 97 pgs: 97 active+clean; 18 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 30 KiB/s rd, 456 MiB/s wr, 122 op/s 2024-04-26T02:02:22.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:21 smithi037 ceph-mon[27906]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 20 KiB/s rd, 307 MiB/s wr, 83 op/s 2024-04-26T02:02:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:21 smithi045 ceph-mon[32520]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 20 KiB/s rd, 307 MiB/s wr, 83 op/s 2024-04-26T02:02:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:21 smithi071 ceph-mon[32578]: pgmap v135: 97 pgs: 97 active+clean; 18 GiB data, 52 GiB used, 1020 GiB / 1.0 TiB avail; 20 KiB/s rd, 307 MiB/s wr, 83 op/s 2024-04-26T02:02:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.012+0000 7f8a39e00640 1 -- 172.21.15.37:0/297623360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 msgr2=0x7f8a3410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.012+0000 7f8a39e00640 1 --2- 172.21.15.37:0/297623360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a3410ec30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009860 tx=0x7f8a2402a360 comp rx=0 tx=0).stop 2024-04-26T02:02:24.012 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 -- 172.21.15.37:0/297623360 shutdown_connections 2024-04-26T02:02:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 --2- 172.21.15.37:0/297623360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a3411b730 0x7f8a3411db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 --2- 172.21.15.37:0/297623360 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a3410ec30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 --2- 172.21.15.37:0/297623360 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a341106f0 0x7f8a3410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 -- 172.21.15.37:0/297623360 >> 172.21.15.37:0/297623360 conn(0x7f8a3406d030 msgr2=0x7f8a3406d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 -- 172.21.15.37:0/297623360 shutdown_connections 2024-04-26T02:02:24.013 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.013+0000 7f8a39e00640 1 -- 172.21.15.37:0/297623360 wait complete. 2024-04-26T02:02:24.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.014+0000 7f8a39e00640 1 Processor -- start 2024-04-26T02:02:24.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 -- start start 2024-04-26T02:02:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a34118d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a341106f0 0x7f8a34111de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 0x7f8a34112320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34121020 con 0x7f8a341106f0 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a34120ea0 con 0x7f8a3411b730 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a39e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341211a0 con 0x7f8a3410e7d0 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a32000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a341106f0 0x7f8a34111de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 0x7f8a34112320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a33400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 0x7f8a34112320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:45116/0 (socket says 172.21.15.37:45116) 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.015+0000 7f8a33400640 1 -- 172.21.15.37:0/102095140 learned_addr learned my addr 172.21.15.37:0/102095140 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.016+0000 7f8a32a00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a34118d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a33400640 1 -- 172.21.15.37:0/102095140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 msgr2=0x7f8a34118d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a33400640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a34118d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a33400640 1 -- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a341106f0 msgr2=0x7f8a34111de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a33400640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a341106f0 0x7f8a34111de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a33400640 1 -- 172.21.15.37:0/102095140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a34112910 con 0x7f8a3411b730 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a32a00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a34118d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.017+0000 7f8a33400640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 0x7f8a34112320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00b3f0 tx=0x7f8a1c00b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.018+0000 7f8a38a00640 1 -- 172.21.15.37:0/102095140 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c015020 con 0x7f8a3411b730 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.018+0000 7f8a38a00640 1 -- 172.21.15.37:0/102095140 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c011ce0 con 0x7f8a3411b730 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.018+0000 7f8a38a00640 1 -- 172.21.15.37:0/102095140 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c010da0 con 0x7f8a3411b730 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.018+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a34112c00 con 0x7f8a3411b730 2024-04-26T02:02:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.018+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a34116b30 con 0x7f8a3411b730 2024-04-26T02:02:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.026+0000 7f8a38a00640 1 -- 172.21.15.37:0/102095140 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a1c018020 con 0x7f8a3411b730 2024-04-26T02:02:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.028+0000 7f8a3cc14740 1 Processor -- start 2024-04-26T02:02:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.030+0000 7f8a3cc14740 1 -- start start 2024-04-26T02:02:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ccbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cd64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564133c31900 con 0x564133cd6a00 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564133bd7ef0 con 0x564133ccb820 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564133c2d8c0 con 0x564133ccc140 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cd64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cd64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51616/0 (socket says 172.21.15.37:51616) 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.031+0000 7f8a32a00640 1 -- 172.21.15.37:0/3306013796 learned_addr learned my addr 172.21.15.37:0/3306013796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.032+0000 7f8a32a00640 1 -- 172.21.15.37:0/3306013796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 msgr2=0x564133ccbc00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.032+0000 7f8a32a00640 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ccbc00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.032+0000 7f8a32a00640 1 -- 172.21.15.37:0/3306013796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 msgr2=0x564133cd8df0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-26T02:02:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.032+0000 7f8a38a00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8a180a3e60 0x7f8a180a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.032+0000 7f8a38a00640 1 -- 172.21.15.37:0/102095140 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8a1c0cfeb0 con 0x7f8a3411b730 2024-04-26T02:02:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.034+0000 7f8a32000640 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.034+0000 7f8a32a00640 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.034+0000 7f8a32a00640 1 -- 172.21.15.37:0/3306013796 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564133944210 con 0x564133ccc140 2024-04-26T02:02:24.034 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.034+0000 7f8a32000640 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T02:02:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.035+0000 7f8a32a00640 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cd64c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009fd0 tx=0x7f8a2c0108a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7bf4b4f60fb4ea server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.035+0000 7f8a23e00640 1 -- 172.21.15.37:0/3306013796 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c004b20 con 0x564133ccc140 2024-04-26T02:02:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.035+0000 7f8a23e00640 1 -- 172.21.15.37:0/3306013796 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c004cc0 con 0x564133ccc140 2024-04-26T02:02:24.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.035+0000 7f8a23e00640 1 -- 172.21.15.37:0/3306013796 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a2c0054f0 con 0x564133ccc140 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3306013796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 msgr2=0x564133cd64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cd64c0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8a2c009fd0 tx=0x7f8a2c0108a0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3306013796 shutdown_connections 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cd64c0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3306013796 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ccbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3306013796 >> 172.21.15.37:0/3306013796 conn(0x564133cc7d30 msgr2=0x564133cc8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a32000640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8a180a3e60 0x7f8a180a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3306013796 shutdown_connections 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.036+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3306013796 wait complete. 2024-04-26T02:02:24.037 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.037+0000 7f8a3cc14740 1 -- start start 2024-04-26T02:02:24.038 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.037+0000 7f8a3cc14740 1 Processor -- start 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.038+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cd3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.038+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cd3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.038+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.038+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cd3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cd3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:51632/0 (socket says 172.21.15.37:51632) 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564133cd9630 con 0x564133cd6a00 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x564133beb310 con 0x564133ccc140 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x564133cd97b0 con 0x564133ccb820 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a3cc14740 10 client.0 _ll_register_callbacks cb 0x564133bb9cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a33400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a33400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34798/0 (socket says 172.21.15.37:34798) 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a33400640 1 -- 172.21.15.37:0/736881260 learned_addr learned my addr 172.21.15.37:0/736881260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:24.039 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a32000640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cd3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a32000640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8a180a3e60 0x7f8a180a6320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a2402a950 tx=0x7f8a24005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a33400640 1 -- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 msgr2=0x564133cd3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a33400640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cd3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a33400640 1 -- 172.21.15.37:0/736881260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 msgr2=0x564133cd3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.039+0000 7f8a32a00640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cd3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-04-26T02:02:24.040 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.041+0000 7f8a33400640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cd3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.041+0000 7f8a33400640 1 -- 172.21.15.37:0/736881260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564133a10620 con 0x564133cd6a00 2024-04-26T02:02:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.041+0000 7f8a32000640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cd3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:24.041 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.041+0000 7f8a33400640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd4480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0a4670 tx=0x7f8a1c0092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06bf9d0200f5e76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.041+0000 7f8a20c00640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.042+0000 7f8a20c00640 1 -- 172.21.15.37:0/736881260 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a4ba0 con 0x564133cd6a00 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.042+0000 7f8a20c00640 1 -- 172.21.15.37:0/736881260 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8a1c0985a0 con 0x564133cd6a00 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.042+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a2c004cc0 con 0x564133cd6a00 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.042+0000 7f8a20c00640 1 -- 172.21.15.37:0/736881260 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8a1c0b5200 con 0x564133cd6a00 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.042+0000 7f8a3cc14740 10 client.15456 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:24.042 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.042+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564133ce5860 con 0x564133cd6a00 2024-04-26T02:02:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.044+0000 7f8a20c00640 1 -- 172.21.15.37:0/736881260 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a2c004cc0 con 0x564133cd6a00 2024-04-26T02:02:24.043 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.044+0000 7f8a20c00640 1 -- 172.21.15.37:0/736881260 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f8a1c09b4a0 con 0x564133cd6a00 2024-04-26T02:02:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a3cc14740 10 client.15456 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a3cc14740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a3cc14740 20 client.15456 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:24.044 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133ce6cd0 0x564133ce9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a3cc14740 4 client.15456 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564133c458b0 con 0x564133ce6cd0 2024-04-26T02:02:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.045+0000 7f8a32a00640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133ce6cd0 0x564133ce9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.046+0000 7f8a32a00640 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133ce6cd0 0x564133ce9100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b40596236574569c server_cookie=89e333ee9f42676c in_seq=0 out_seq=0 2024-04-26T02:02:24.045 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.046+0000 7f8a20c00640 10 client.15456 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.047+0000 7f8a20c00640 1 -- 172.21.15.37:0/736881260 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564133c458b0 con 0x564133ce6cd0 2024-04-26T02:02:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.047+0000 7f8a20c00640 10 client.15456 handle_command_reply: tid=0 2024-04-26T02:02:24.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.047+0000 7f8a3cc14740 1 client.15456 shutdown 2024-04-26T02:02:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.047+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 msgr2=0x564133cd4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.047 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd4480 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c0a4670 tx=0x7f8a1c0092f0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 shutdown_connections 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133ce6cd0 0x564133ce9100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133cd4480 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cd3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/736881260 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cd3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 >> 172.21.15.37:0/736881260 conn(0x564133cc7d30 msgr2=0x564133ccda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a20c00640 0 client.15456 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 shutdown_connections 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.048+0000 7f8a3cc14740 1 -- 172.21.15.37:0/736881260 wait complete. 2024-04-26T02:02:24.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.049+0000 7f8a3cc14740 20 client.15456 trim_cache size 0 max 16384 2024-04-26T02:02:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.069+0000 7f8a3cc14740 1 Processor -- start 2024-04-26T02:02:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.069+0000 7f8a3cc14740 1 -- start start 2024-04-26T02:02:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ce24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133ceea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c31f0 con 0x564133cd6a00 2024-04-26T02:02:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c5a70 con 0x564133ccb820 2024-04-26T02:02:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c3860 con 0x564133ccc140 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ce24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a32a00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ce24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:45144/0 (socket says 172.21.15.37:45144) 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a32a00640 1 -- 172.21.15.37:0/1176181557 learned_addr learned my addr 172.21.15.37:0/1176181557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a33400640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133ceea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.070+0000 7f8a32000640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a33400640 1 -- 172.21.15.37:0/1176181557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 msgr2=0x564133cce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a33400640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a33400640 1 -- 172.21.15.37:0/1176181557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 msgr2=0x564133ce24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a33400640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ce24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.072 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a33400640 1 -- 172.21.15.37:0/1176181557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564133cd9160 con 0x564133cd6a00 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a32a00640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ce24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a32000640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a33400640 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133ceea30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00bdb0 tx=0x7f8a1c0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad3fa11423d48aae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.071+0000 7f8a23400640 1 -- 172.21.15.37:0/1176181557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0a8030 con 0x564133cd6a00 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a23400640 1 -- 172.21.15.37:0/1176181557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c09bf30 con 0x564133cd6a00 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a23400640 1 -- 172.21.15.37:0/1176181557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1c0cd7c0 con 0x564133cd6a00 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 -- 172.21.15.37:0/1176181557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 msgr2=0x564133ceea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133ceea30 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00bdb0 tx=0x7f8a1c0047d0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 -- 172.21.15.37:0/1176181557 shutdown_connections 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133ceea30 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccc140 0x564133cce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/1176181557 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccb820 0x564133ce24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.072+0000 7f8a3cc14740 1 -- 172.21.15.37:0/1176181557 >> 172.21.15.37:0/1176181557 conn(0x564133cc7d30 msgr2=0x564133cc8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.073+0000 7f8a3cc14740 1 -- 172.21.15.37:0/1176181557 shutdown_connections 2024-04-26T02:02:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.073+0000 7f8a3cc14740 1 -- 172.21.15.37:0/1176181557 wait complete. 2024-04-26T02:02:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.073+0000 7f8a3cc14740 1 -- start start 2024-04-26T02:02:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.073+0000 7f8a3cc14740 1 Processor -- start 2024-04-26T02:02:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cdbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cdc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133d4b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c3f30 con 0x564133cd6a00 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c53a0 con 0x564133ccc140 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8a341c4600 con 0x564133ccb820 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.074+0000 7f8a3cc14740 10 client.0 _ll_register_callbacks cb 0x7f8a1c09b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cdc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cdc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:45160/0 (socket says 172.21.15.37:45160) 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 -- 172.21.15.37:0/3920510991 learned_addr learned my addr 172.21.15.37:0/3920510991 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32a00640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cdbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a33400640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133d4b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 -- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 msgr2=0x564133cdbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cdbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 -- 172.21.15.37:0/3920510991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 msgr2=0x564133d4b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133d4b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.075+0000 7f8a32000640 1 -- 172.21.15.37:0/3920510991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564133d4b7e0 con 0x564133ccc140 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.076+0000 7f8a32000640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cdc100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009860 tx=0x7f8a240056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bf7bc335aec2652 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.076+0000 7f8a20c00640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.076+0000 7f8a20c00640 1 -- 172.21.15.37:0/3920510991 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24037040 con 0x564133ccc140 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.076+0000 7f8a20c00640 1 -- 172.21.15.37:0/3920510991 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f8a24003b40 con 0x564133ccc140 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.076+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a1c09bf30 con 0x564133ccc140 2024-04-26T02:02:24.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.076+0000 7f8a20c00640 1 -- 172.21.15.37:0/3920510991 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f8a24030e40 con 0x564133ccc140 2024-04-26T02:02:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.077+0000 7f8a3cc14740 10 client.25550 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.077+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x564133d4ca00 con 0x564133ccc140 2024-04-26T02:02:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.078+0000 7f8a20c00640 1 -- 172.21.15.37:0/3920510991 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a1c09bf30 con 0x564133ccc140 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.078+0000 7f8a20c00640 1 -- 172.21.15.37:0/3920510991 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f8a24049070 con 0x564133ccc140 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.078+0000 7f8a3cc14740 10 client.25550 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.079+0000 7f8a3cc14740 10 client.25550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.079+0000 7f8a3cc14740 20 client.25550 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.079+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133d4d200 0x564133d4f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.079+0000 7f8a3cc14740 4 client.25550 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.079+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564133c3f320 con 0x564133d4d200 2024-04-26T02:02:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.080+0000 7f8a32a00640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133d4d200 0x564133d4f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.085+0000 7f8a32a00640 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133d4d200 0x564133d4f620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aacfd2c838f20236 server_cookie=d2abe418db6c22d0 in_seq=0 out_seq=0 2024-04-26T02:02:24.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.085+0000 7f8a20c00640 10 client.25550 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:24.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.098+0000 7f8a20c00640 1 -- 172.21.15.37:0/3920510991 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564133c3f320 con 0x564133d4d200 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.098+0000 7f8a20c00640 10 client.25550 handle_command_reply: tid=0 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.098+0000 7f8a3cc14740 1 client.25550 shutdown 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.098+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 msgr2=0x564133cdc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.098+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cdc100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8a24009860 tx=0x7f8a240056d0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 shutdown_connections 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x564133d4d200 0x564133d4f620 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564133cd6a00 0x564133d4b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x564133ccc140 0x564133cdc100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a20c00640 0 client.25550 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 --2- 172.21.15.37:0/3920510991 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x564133ccb820 0x564133cdbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 >> 172.21.15.37:0/3920510991 conn(0x564133cc7d30 msgr2=0x564133cced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 shutdown_connections 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 1 -- 172.21.15.37:0/3920510991 wait complete. 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.099+0000 7f8a3cc14740 20 client.25550 trim_cache size 0 max 16384 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:02:24.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:02:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8a180a3e60 msgr2=0x7f8a180a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8a180a3e60 0x7f8a180a6320 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f8a2402a950 tx=0x7f8a24005f90 comp rx=0 tx=0).stop 2024-04-26T02:02:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 msgr2=0x7f8a34112320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 0x7f8a34112320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c00b3f0 tx=0x7f8a1c00b8b0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 shutdown_connections 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f8a180a3e60 0x7f8a180a6320 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8a3411b730 0x7f8a34112320 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a341106f0 0x7f8a34111de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 --2- 172.21.15.37:0/102095140 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8a3410e7d0 0x7f8a34118d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 >> 172.21.15.37:0/102095140 conn(0x7f8a3406d030 msgr2=0x7f8a3411a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 shutdown_connections 2024-04-26T02:02:24.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:24.100+0000 7f8a39e00640 1 -- 172.21.15.37:0/102095140 wait complete. 2024-04-26T02:02:24.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:23 smithi037 ceph-mon[27906]: pgmap v136: 97 pgs: 97 active+clean; 19 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 22 KiB/s rd, 458 MiB/s wr, 122 op/s 2024-04-26T02:02:24.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:23 smithi045 ceph-mon[32520]: pgmap v136: 97 pgs: 97 active+clean; 19 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 22 KiB/s rd, 458 MiB/s wr, 122 op/s 2024-04-26T02:02:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:23 smithi071 ceph-mon[32578]: pgmap v136: 97 pgs: 97 active+clean; 19 GiB data, 58 GiB used, 1015 GiB / 1.0 TiB avail; 22 KiB/s rd, 458 MiB/s wr, 122 op/s 2024-04-26T02:02:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:25 smithi037 ceph-mon[27906]: pgmap v137: 97 pgs: 97 active+clean; 20 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 12 KiB/s rd, 369 MiB/s wr, 95 op/s 2024-04-26T02:02:26.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:25 smithi045 ceph-mon[32520]: pgmap v137: 97 pgs: 97 active+clean; 20 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 12 KiB/s rd, 369 MiB/s wr, 95 op/s 2024-04-26T02:02:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:25 smithi071 ceph-mon[32578]: pgmap v137: 97 pgs: 97 active+clean; 20 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 12 KiB/s rd, 369 MiB/s wr, 95 op/s 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 rewriters = 153286.39 kB/sec 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 rewriters = 153284.50 kB/sec 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 153286.39 kB/sec 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 153286.39 kB/sec 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 153286.39 kB/sec 2024-04-26T02:02:26.505 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 10485760.00 kB 2024-04-26T02:02:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:27 smithi037 ceph-mon[27906]: pgmap v138: 97 pgs: 97 active+clean; 20 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 12 KiB/s rd, 338 MiB/s wr, 87 op/s 2024-04-26T02:02:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:27 smithi045 ceph-mon[32520]: pgmap v138: 97 pgs: 97 active+clean; 20 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 12 KiB/s rd, 338 MiB/s wr, 87 op/s 2024-04-26T02:02:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:27 smithi071 ceph-mon[32578]: pgmap v138: 97 pgs: 97 active+clean; 20 GiB data, 59 GiB used, 1014 GiB / 1.0 TiB avail; 12 KiB/s rd, 338 MiB/s wr, 87 op/s 2024-04-26T02:02:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:29 smithi037 ceph-mon[27906]: pgmap v139: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 462 MiB/s wr, 122 op/s 2024-04-26T02:02:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:29 smithi045 ceph-mon[32520]: pgmap v139: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 462 MiB/s wr, 122 op/s 2024-04-26T02:02:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:29 smithi071 ceph-mon[32578]: pgmap v139: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 15 KiB/s rd, 462 MiB/s wr, 122 op/s 2024-04-26T02:02:30.519 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:02:30.519 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 readers = 5873020.50 kB/sec 2024-04-26T02:02:30.519 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 readers = 5869919.56 kB/sec 2024-04-26T02:02:30.519 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 5873020.50 kB/sec 2024-04-26T02:02:30.519 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 5873020.50 kB/sec 2024-04-26T02:02:30.520 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 5873020.50 kB/sec 2024-04-26T02:02:30.520 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 10485760.00 kB 2024-04-26T02:02:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:31 smithi037 ceph-mon[27906]: pgmap v140: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 294 MiB/s wr, 78 op/s 2024-04-26T02:02:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:31 smithi045 ceph-mon[32520]: pgmap v140: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 294 MiB/s wr, 78 op/s 2024-04-26T02:02:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:31 smithi071 ceph-mon[32578]: pgmap v140: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.7 KiB/s rd, 294 MiB/s wr, 78 op/s 2024-04-26T02:02:34.204 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:33 smithi071 ceph-mon[32578]: pgmap v141: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 294 MiB/s wr, 79 op/s 2024-04-26T02:02:34.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:33 smithi037 ceph-mon[27906]: pgmap v141: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 294 MiB/s wr, 79 op/s 2024-04-26T02:02:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:33 smithi045 ceph-mon[32520]: pgmap v141: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 10 KiB/s rd, 294 MiB/s wr, 79 op/s 2024-04-26T02:02:35.403 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:02:35.412 INFO:tasks.workunit.client.0.smithi037.stdout: Children see throughput for 1 re-readers = 3777815.00 kB/sec 2024-04-26T02:02:35.412 INFO:tasks.workunit.client.0.smithi037.stdout: Parent sees throughput for 1 re-readers = 3776591.79 kB/sec 2024-04-26T02:02:35.412 INFO:tasks.workunit.client.0.smithi037.stdout: Min throughput per process = 3777815.00 kB/sec 2024-04-26T02:02:35.412 INFO:tasks.workunit.client.0.smithi037.stdout: Max throughput per process = 3777815.00 kB/sec 2024-04-26T02:02:35.413 INFO:tasks.workunit.client.0.smithi037.stdout: Avg throughput per process = 3777815.00 kB/sec 2024-04-26T02:02:35.413 INFO:tasks.workunit.client.0.smithi037.stdout: Min xfer = 10485760.00 kB 2024-04-26T02:02:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:35 smithi037 ceph-mon[27906]: pgmap v142: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 143 MiB/s wr, 40 op/s 2024-04-26T02:02:36.256 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:35 smithi045 ceph-mon[32520]: pgmap v142: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 143 MiB/s wr, 40 op/s 2024-04-26T02:02:36.404 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:35 smithi071 ceph-mon[32578]: pgmap v142: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 143 MiB/s wr, 40 op/s 2024-04-26T02:02:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 -- 172.21.15.37:0/1502879813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c48104220 msgr2=0x7f0c48104600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1502879813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c48104220 0x7f0c48104600 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009ca0 tx=0x7f0c3802a8c0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 -- 172.21.15.37:0/1502879813 shutdown_connections 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1502879813 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c481092f0 0x7f0c48110ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1502879813 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c48104bd0 0x7f0c48108c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1502879813 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c48104220 0x7f0c48104600 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.251+0000 7f0c4d800640 1 -- 172.21.15.37:0/1502879813 >> 172.21.15.37:0/1502879813 conn(0x7f0c480ffa60 msgr2=0x7f0c48101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.252+0000 7f0c4d800640 1 -- 172.21.15.37:0/1502879813 shutdown_connections 2024-04-26T02:02:37.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.252+0000 7f0c4d800640 1 -- 172.21.15.37:0/1502879813 wait complete. 2024-04-26T02:02:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 Processor -- start 2024-04-26T02:02:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 -- start start 2024-04-26T02:02:37.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c48104220 0x7f0c481a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 0x7f0c481a0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4819a620 0x7f0c4819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c48114260 con 0x7f0c481092f0 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481140e0 con 0x7f0c4819a620 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c4d800640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481143e0 con 0x7f0c48104220 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c47400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 0x7f0c481a0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.253+0000 7f0c47400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 0x7f0c481a0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57930/0 (socket says 172.21.15.37:57930) 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 -- 172.21.15.37:0/1693600862 learned_addr learned my addr 172.21.15.37:0/1693600862 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47e00640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c48104220 0x7f0c481a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c4ce00640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4819a620 0x7f0c4819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 -- 172.21.15.37:0/1693600862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c48104220 msgr2=0x7f0c481a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c48104220 0x7f0c481a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 -- 172.21.15.37:0/1693600862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4819a620 msgr2=0x7f0c4819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4819a620 0x7f0c4819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 -- 172.21.15.37:0/1693600862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4819b140 con 0x7f0c481092f0 2024-04-26T02:02:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.254+0000 7f0c47400640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 0x7f0c481a0960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000e9c0 tx=0x7f0c3000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.255+0000 7f0c44c00640 1 -- 172.21.15.37:0/1693600862 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3000cd80 con 0x7f0c481092f0 2024-04-26T02:02:37.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.255+0000 7f0c44c00640 1 -- 172.21.15.37:0/1693600862 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c300054d0 con 0x7f0c481092f0 2024-04-26T02:02:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.255+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4819b430 con 0x7f0c481092f0 2024-04-26T02:02:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.255+0000 7f0c44c00640 1 -- 172.21.15.37:0/1693600862 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30010430 con 0x7f0c481092f0 2024-04-26T02:02:37.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.255+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c481a7680 con 0x7f0c481092f0 2024-04-26T02:02:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.258+0000 7f0c51b89740 1 Processor -- start 2024-04-26T02:02:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.258+0000 7f0c44c00640 1 -- 172.21.15.37:0/1693600862 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0c30004fd0 con 0x7f0c481092f0 2024-04-26T02:02:37.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.258+0000 7f0c51b89740 1 -- start start 2024-04-26T02:02:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6726d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b6731650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b6733f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c51b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5579b668c900 con 0x5579b67272d0 2024-04-26T02:02:37.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c51b89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579b6632ef0 con 0x5579b6731b90 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c51b89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579b66888c0 con 0x5579b67269b0 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b6731650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c4ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6726d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c4ce00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6726d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:44988/0 (socket says 172.21.15.37:44988) 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c4ce00640 1 -- 172.21.15.37:0/66106296 learned_addr learned my addr 172.21.15.37:0/66106296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.259+0000 7f0c47400640 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b6733f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c47e00640 1 -- 172.21.15.37:0/66106296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 msgr2=0x5579b6726d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c47e00640 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6726d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c47e00640 1 -- 172.21.15.37:0/66106296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 msgr2=0x5579b6733f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c47e00640 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b6733f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c47e00640 1 -- 172.21.15.37:0/66106296 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b639f210 con 0x5579b67272d0 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c4ce00640 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6726d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c47e00640 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b6731650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c38000f80 tx=0x7f0c380093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5953783588e248e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:37.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c44c00640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0c2c0a3d80 0x7f0c2c0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c3d600640 1 -- 172.21.15.37:0/66106296 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38031cf0 con 0x5579b67272d0 2024-04-26T02:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c3d600640 1 -- 172.21.15.37:0/66106296 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c38036030 con 0x5579b67272d0 2024-04-26T02:02:37.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c44c00640 1 -- 172.21.15.37:0/1693600862 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0c300d0b80 con 0x7f0c481092f0 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.260+0000 7f0c4ce00640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0c2c0a3d80 0x7f0c2c0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c3d600640 1 -- 172.21.15.37:0/66106296 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c38004040 con 0x5579b67272d0 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 -- 172.21.15.37:0/66106296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 msgr2=0x5579b6731650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b6731650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0c38000f80 tx=0x7f0c380093f0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c4ce00640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0c2c0a3d80 0x7f0c2c0a6240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c400124a0 tx=0x7f0c40005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 -- 172.21.15.37:0/66106296 shutdown_connections 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b6733f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b6731650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 --2- 172.21.15.37:0/66106296 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6726d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.261+0000 7f0c51b89740 1 -- 172.21.15.37:0/66106296 >> 172.21.15.37:0/66106296 conn(0x5579b6722ec0 msgr2=0x5579b67232a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.262+0000 7f0c51b89740 1 -- 172.21.15.37:0/66106296 shutdown_connections 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.262+0000 7f0c51b89740 1 -- 172.21.15.37:0/66106296 wait complete. 2024-04-26T02:02:37.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.262+0000 7f0c51b89740 1 -- start start 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.262+0000 7f0c51b89740 1 Processor -- start 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67269b0 0x5579b672eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67272d0 0x5579b672f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b672f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5579b67345e0 con 0x5579b67269b0 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x5579b6646310 con 0x5579b6731b90 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x5579b6734760 con 0x5579b67272d0 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c51b89740 10 client.0 _ll_register_callbacks cb 0x5579b6614cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67269b0 0x5579b672eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67272d0 0x5579b672f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67269b0 0x5579b672eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57948/0 (socket says 172.21.15.37:57948) 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47e00640 1 -- 172.21.15.37:0/1142777022 learned_addr learned my addr 172.21.15.37:0/1142777022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c4ce00640 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b672f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47400640 1 -- 172.21.15.37:0/1142777022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 msgr2=0x5579b672f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47400640 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b672f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47400640 1 -- 172.21.15.37:0/1142777022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67269b0 msgr2=0x5579b672eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47400640 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67269b0 0x5579b672eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.263+0000 7f0c47400640 1 -- 172.21.15.37:0/1142777022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b646b620 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c47400640 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67272d0 0x5579b672f030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ce2a0 tx=0x7f0c300049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd86a6034fe910c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c36000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c36000640 1 -- 172.21.15.37:0/1142777022 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3009c630 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c36000640 1 -- 172.21.15.37:0/1142777022 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0c3009adf0 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c38036030 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c36000640 1 -- 172.21.15.37:0/1142777022 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0c3009c450 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c51b89740 10 client.25627 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5579b67408a0 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.264+0000 7f0c36000640 1 -- 172.21.15.37:0/1142777022 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c38036030 con 0x5579b67272d0 2024-04-26T02:02:37.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c36000640 1 -- 172.21.15.37:0/1142777022 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f0c300ce520 con 0x5579b67272d0 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c51b89740 10 client.25627 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c51b89740 10 client.25627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c51b89740 20 client.25627 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b6741d60 0x5579b6744150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c51b89740 4 client.25627 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579b66a08b0 con 0x5579b6741d60 2024-04-26T02:02:37.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.265+0000 7f0c47e00640 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b6741d60 0x5579b6744150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.266+0000 7f0c47e00640 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b6741d60 0x5579b6744150 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecd9d7e77077a481 server_cookie=c199bebba30e6bc6 in_seq=0 out_seq=0 2024-04-26T02:02:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.266+0000 7f0c36000640 10 client.25627 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.266+0000 7f0c36000640 1 -- 172.21.15.37:0/1142777022 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579b66a08b0 con 0x5579b6741d60 2024-04-26T02:02:37.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.266+0000 7f0c36000640 10 client.25627 handle_command_reply: tid=0 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.266+0000 7f0c51b89740 1 client.25627 shutdown 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67272d0 msgr2=0x5579b672f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67272d0 0x5579b672f030 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0c300ce2a0 tx=0x7f0c300049b0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 shutdown_connections 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b6741d60 0x5579b6744150 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b672f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67272d0 0x5579b672f030 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c36000640 0 client.25627 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1142777022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67269b0 0x5579b672eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 >> 172.21.15.37:0/1142777022 conn(0x5579b6722ec0 msgr2=0x5579b6728bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 shutdown_connections 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 1 -- 172.21.15.37:0/1142777022 wait complete. 2024-04-26T02:02:37.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.267+0000 7f0c51b89740 20 client.25627 trim_cache size 0 max 16384 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 Processor -- start 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 -- start start 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b673df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b673d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b67227a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481adab0 con 0x5579b67272d0 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481b0310 con 0x5579b6731b90 2024-04-26T02:02:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c51b89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481ae100 con 0x5579b67269b0 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c47400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b673d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c47400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b673d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57962/0 (socket says 172.21.15.37:57962) 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c47400640 1 -- 172.21.15.37:0/1275029833 learned_addr learned my addr 172.21.15.37:0/1275029833 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.283+0000 7f0c47e00640 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b673df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c4ce00640 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b67227a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c47400640 1 -- 172.21.15.37:0/1275029833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 msgr2=0x5579b673df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c47400640 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b673df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c47400640 1 -- 172.21.15.37:0/1275029833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 msgr2=0x5579b67227a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c47400640 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b67227a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c47400640 1 -- 172.21.15.37:0/1275029833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579b672dac0 con 0x5579b67272d0 2024-04-26T02:02:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c47400640 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b673d4a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009360 tx=0x7f0c3009c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7c173d19c337f54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c3cc00640 1 -- 172.21.15.37:0/1275029833 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3009c630 con 0x5579b67272d0 2024-04-26T02:02:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c3cc00640 1 -- 172.21.15.37:0/1275029833 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c300046c0 con 0x5579b67272d0 2024-04-26T02:02:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.284+0000 7f0c3cc00640 1 -- 172.21.15.37:0/1275029833 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3009adf0 con 0x5579b67272d0 2024-04-26T02:02:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 -- 172.21.15.37:0/1275029833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 msgr2=0x5579b673d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b673d4a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009360 tx=0x7f0c3009c980 comp rx=0 tx=0).stop 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 -- 172.21.15.37:0/1275029833 shutdown_connections 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b6731b90 0x5579b67227a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b67272d0 0x5579b673d4a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 --2- 172.21.15.37:0/1275029833 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b673df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 -- 172.21.15.37:0/1275029833 >> 172.21.15.37:0/1275029833 conn(0x5579b6722ec0 msgr2=0x5579b6728bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 -- 172.21.15.37:0/1275029833 shutdown_connections 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 -- 172.21.15.37:0/1275029833 wait complete. 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.285+0000 7f0c51b89740 1 -- start start 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 Processor -- start 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6737720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 0x5579b6737c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b6731b90 0x5579b67a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481ae7d0 con 0x5579b6731b90 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481afc40 con 0x5579b67272d0 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481aeea0 con 0x5579b67269b0 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c51b89740 10 client.0 _ll_register_callbacks cb 0x7f0c3000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6737720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c47400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 0x5579b6737c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c4ce00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b6731b90 0x5579b67a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c47400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 0x5579b6737c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:49240/0 (socket says 172.21.15.37:49240) 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c47400640 1 -- 172.21.15.37:0/3184733618 learned_addr learned my addr 172.21.15.37:0/3184733618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.286+0000 7f0c47e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6737720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:45012/0 (socket says 172.21.15.37:45012) 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47e00640 1 -- 172.21.15.37:0/3184733618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 msgr2=0x5579b6737c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47e00640 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 0x5579b6737c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47e00640 1 -- 172.21.15.37:0/3184733618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b6731b90 msgr2=0x5579b67a72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47e00640 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b6731b90 0x5579b67a72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47e00640 1 -- 172.21.15.37:0/3184733618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579b6727100 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47400640 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 0x5579b6737c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c47e00640 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6737720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5579b67449b0 tx=0x7f0c3802f530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7083601bc029d7c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c36000640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c36000640 1 -- 172.21.15.37:0/3184733618 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3805b060 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5579b669a320 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.287+0000 7f0c36000640 1 -- 172.21.15.37:0/3184733618 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f0c38031cf0 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c36000640 1 -- 172.21.15.37:0/3184733618 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f0c3804a040 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 10 client.25633 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x5579b67a8720 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c36000640 1 -- 172.21.15.37:0/3184733618 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5579b669a320 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c36000640 1 -- 172.21.15.37:0/3184733618 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f0c38043070 con 0x5579b67269b0 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 10 client.25633 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 10 client.25633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 20 client.25633 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b67a8b70 0x5579b67aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 4 client.25633 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.288+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5579b6694190 con 0x5579b67a8b70 2024-04-26T02:02:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.289+0000 7f0c47400640 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b67a8b70 0x5579b67aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.289+0000 7f0c47400640 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b67a8b70 0x5579b67aaf60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e087cb91646fc35c server_cookie=c560a575029e0dc5 in_seq=0 out_seq=0 2024-04-26T02:02:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.289+0000 7f0c36000640 10 client.25633 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c36000640 1 -- 172.21.15.37:0/3184733618 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5579b6694190 con 0x5579b67a8b70 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c36000640 10 client.25633 handle_command_reply: tid=0 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c51b89740 1 client.25633 shutdown 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 msgr2=0x5579b6737720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c51b89740 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6737720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x5579b67449b0 tx=0x7f0c3802f530 comp rx=0 tx=0).stop 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 shutdown_connections 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.298+0000 7f0c51b89740 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x5579b67a8b70 0x5579b67aaf60 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579b6731b90 0x5579b67a72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x5579b67272d0 0x5579b6737c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 1 --2- 172.21.15.37:0/3184733618 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x5579b67269b0 0x5579b6737720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c36000640 0 client.25633 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 >> 172.21.15.37:0/3184733618 conn(0x5579b6722ec0 msgr2=0x5579b6728bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 shutdown_connections 2024-04-26T02:02:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 1 -- 172.21.15.37:0/3184733618 wait complete. 2024-04-26T02:02:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c51b89740 20 client.25633 trim_cache size 0 max 16384 2024-04-26T02:02:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:02:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:02:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:02:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0c2c0a3d80 msgr2=0x7f0c2c0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0c2c0a3d80 0x7f0c2c0a6240 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0c400124a0 tx=0x7f0c40005f90 comp rx=0 tx=0).stop 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 msgr2=0x7f0c481a0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.299+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 0x7f0c481a0960 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0c3000e9c0 tx=0x7f0c3000ee80 comp rx=0 tx=0).stop 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 shutdown_connections 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f0c2c0a3d80 0x7f0c2c0a6240 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f0c4819a620 0x7f0c4819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c481092f0 0x7f0c481a0960 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 --2- 172.21.15.37:0/1693600862 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f0c48104220 0x7f0c481a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 >> 172.21.15.37:0/1693600862 conn(0x7f0c480ffa60 msgr2=0x7f0c48100de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 shutdown_connections 2024-04-26T02:02:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:37.300+0000 7f0c4d800640 1 -- 172.21.15.37:0/1693600862 wait complete. 2024-04-26T02:02:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:37 smithi045 ceph-mon[32520]: pgmap v143: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 124 MiB/s wr, 35 op/s 2024-04-26T02:02:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:37 smithi071 ceph-mon[32578]: pgmap v143: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 124 MiB/s wr, 35 op/s 2024-04-26T02:02:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:37 smithi037 ceph-mon[27906]: pgmap v143: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.3 KiB/s rd, 124 MiB/s wr, 35 op/s 2024-04-26T02:02:40.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:39 smithi045 ceph-mon[32520]: pgmap v144: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 124 MiB/s wr, 38 op/s 2024-04-26T02:02:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:39 smithi071 ceph-mon[32578]: pgmap v144: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 124 MiB/s wr, 38 op/s 2024-04-26T02:02:40.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:39 smithi037 ceph-mon[27906]: pgmap v144: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 8.2 KiB/s rd, 124 MiB/s wr, 38 op/s 2024-04-26T02:02:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:41 smithi045 ceph-mon[32520]: pgmap v145: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.4 KiB/s wr, 4 op/s 2024-04-26T02:02:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:41 smithi071 ceph-mon[32578]: pgmap v145: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.4 KiB/s wr, 4 op/s 2024-04-26T02:02:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:41 smithi037 ceph-mon[27906]: pgmap v145: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 6.4 KiB/s wr, 4 op/s 2024-04-26T02:02:43.640 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:02:43.640 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:02:43.640 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-04-26T02:02:43.640 INFO:tasks.workunit.client.0.smithi037.stdout:iozone test complete. 2024-04-26T02:02:43.646 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T02:02:43.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T02:02:43.796 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-04-26T02:02:43.797 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T02:02:44.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:43 smithi037 ceph-mon[27906]: pgmap v146: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-04-26T02:02:44.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:43 smithi045 ceph-mon[32520]: pgmap v146: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-04-26T02:02:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:43 smithi071 ceph-mon[32578]: pgmap v146: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 7.7 KiB/s rd, 8.7 KiB/s wr, 5 op/s 2024-04-26T02:02:45.563 DEBUG:teuthology.parallel:result is None 2024-04-26T02:02:45.563 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T02:02:45.648 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T02:02:45.649 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-26T02:02:45.661 INFO:tasks.quiescer:joining Quiescers 2024-04-26T02:02:45.661 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-26T02:02:45.661 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:02:46.115 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:02:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:45 smithi045 ceph-mon[32520]: pgmap v147: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-26T02:02:46.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:45 smithi037 ceph-mon[27906]: pgmap v147: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-26T02:02:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:45 smithi071 ceph-mon[32578]: pgmap v147: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-26T02:02:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:47 smithi045 ceph-mon[32520]: pgmap v148: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-26T02:02:48.291 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:47 smithi037 ceph-mon[27906]: pgmap v148: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-26T02:02:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:47 smithi071 ceph-mon[32578]: pgmap v148: 97 pgs: 97 active+clean; 21 GiB data, 64 GiB used, 1009 GiB / 1.0 TiB avail; 3.4 KiB/s rd, 4.7 KiB/s wr, 4 op/s 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 -- 172.21.15.37:0/503209138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 msgr2=0x7ffb98111b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 --2- 172.21.15.37:0/503209138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb98111b70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffb9000b0d0 tx=0x7ffb9000b3e0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 -- 172.21.15.37:0/503209138 shutdown_connections 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 --2- 172.21.15.37:0/503209138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb98111b70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 --2- 172.21.15.37:0/503209138 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb98104ec0 0x7ffb981052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 --2- 172.21.15.37:0/503209138 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 0x7ffb981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 -- 172.21.15.37:0/503209138 >> 172.21.15.37:0/503209138 conn(0x7ffb98100750 msgr2=0x7ffb98102b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 -- 172.21.15.37:0/503209138 shutdown_connections 2024-04-26T02:02:48.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.879+0000 7ffb9d600700 1 -- 172.21.15.37:0/503209138 wait complete. 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 Processor -- start 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 -- start start 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb98104ec0 0x7ffb9819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 0x7ffb9819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb9819fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98114ea0 con 0x7ffb98109fa0 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98114d20 con 0x7ffb98104ec0 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.881+0000 7ffb9d600700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7ffb98115020 con 0x7ffb98105870 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb98104ec0 0x7ffb9819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb97400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb9819fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 0x7ffb9819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb97400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb9819fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41496/0 (socket says 172.21.15.37:41496) 2024-04-26T02:02:48.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb97400700 1 -- 172.21.15.37:0/2730526385 learned_addr learned my addr 172.21.15.37:0/2730526385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96000700 1 -- 172.21.15.37:0/2730526385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb98104ec0 msgr2=0x7ffb9819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96000700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb98104ec0 0x7ffb9819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96000700 1 -- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 msgr2=0x7ffb9819fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96000700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb9819fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb96000700 1 -- 172.21.15.37:0/2730526385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb981a0310 con 0x7ffb98105870 2024-04-26T02:02:48.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.882+0000 7ffb97400700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb9819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.883+0000 7ffb96000700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 0x7ffb9819b890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e9c0 tx=0x7ffb8400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.883+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84004220 con 0x7ffb98105870 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.883+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb981a0600 con 0x7ffb98105870 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.883+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb840043c0 con 0x7ffb98105870 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.883+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84005020 con 0x7ffb98105870 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.883+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb981a7e80 con 0x7ffb98105870 2024-04-26T02:02:48.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.884+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb98069730 con 0x7ffb98105870 2024-04-26T02:02:48.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.886+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb84004750 con 0x7ffb98105870 2024-04-26T02:02:48.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.888+0000 7ffb8f400700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffb7c0a3db0 0x7ffb7c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:48.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.888+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7ffb840d0720 con 0x7ffb98105870 2024-04-26T02:02:48.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.888+0000 7ffb96a00700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffb7c0a3db0 0x7ffb7c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:48.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.889+0000 7ffb96a00700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffb7c0a3db0 0x7ffb7c0a6270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009c60 tx=0x7ffb88009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:48.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:48.892+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb8409cd90 con 0x7ffb98105870 2024-04-26T02:02:49.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.179+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb9819c660 con 0x7ffb98105870 2024-04-26T02:02:49.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.180+0000 7ffb8f400700 1 -- 172.21.15.37:0/2730526385 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7ffb8409c690 con 0x7ffb98105870 2024-04-26T02:02:49.179 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:02:49.180 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffb7c0a3db0 msgr2=0x7ffb7c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffb7c0a3db0 0x7ffb7c0a6270 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009c60 tx=0x7ffb88009260 comp rx=0 tx=0).stop 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 msgr2=0x7ffb9819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 0x7ffb9819b890 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb8400e9c0 tx=0x7ffb8400ecd0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 shutdown_connections 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb98109fa0 0x7ffb9819fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7ffb7c0a3db0 0x7ffb7c0a6270 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7ffb98104ec0 0x7ffb9819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 --2- 172.21.15.37:0/2730526385 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7ffb98105870 0x7ffb9819b890 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 >> 172.21.15.37:0/2730526385 conn(0x7ffb98100750 msgr2=0x7ffb98101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.183+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 shutdown_connections 2024-04-26T02:02:49.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:49.184+0000 7ffb9d600700 1 -- 172.21.15.37:0/2730526385 wait complete. 2024-04-26T02:02:49.185 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:02:50.152 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24514 quiesce db --cancel --all' 2024-04-26T02:02:50.153 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24514 quiesce db --cancel --all 2024-04-26T02:02:50.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:49 smithi045 ceph-mon[32520]: pgmap v149: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.2 KiB/s wr, 219 op/s 2024-04-26T02:02:50.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:49 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2730526385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:49 smithi071 ceph-mon[32578]: pgmap v149: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.2 KiB/s wr, 219 op/s 2024-04-26T02:02:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:49 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2730526385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 -- 172.21.15.37:0/2101864234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c1014e0 msgr2=0x7fab7c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 --2- 172.21.15.37:0/2101864234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c1014e0 0x7fab7c10e1a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fab68009900 tx=0x7fab6802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 -- 172.21.15.37:0/2101864234 shutdown_connections 2024-04-26T02:02:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 --2- 172.21.15.37:0/2101864234 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab7c10e6e0 0x7fab7c110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 --2- 172.21.15.37:0/2101864234 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c1014e0 0x7fab7c10e1a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 --2- 172.21.15.37:0/2101864234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c100b30 0x7fab7c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 -- 172.21.15.37:0/2101864234 >> 172.21.15.37:0/2101864234 conn(0x7fab7c0fc730 msgr2=0x7fab7c0feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 -- 172.21.15.37:0/2101864234 shutdown_connections 2024-04-26T02:02:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.443+0000 7fab83000640 1 -- 172.21.15.37:0/2101864234 wait complete. 2024-04-26T02:02:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.444+0000 7fab83000640 1 Processor -- start 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 -- start start 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab7c100b30 0x7fab7c1a0460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 0x7fab7c1a09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c10e6e0 0x7fab7c19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c114270 con 0x7fab7c1014e0 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1140f0 con 0x7fab7c10e6e0 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab83000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1143f0 con 0x7fab7c100b30 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab81200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 0x7fab7c1a09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab81200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 0x7fab7c1a09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43524/0 (socket says 172.21.15.37:43524) 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.445+0000 7fab81200640 1 -- 172.21.15.37:0/1362051265 learned_addr learned my addr 172.21.15.37:0/1362051265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81c00640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab7c100b30 0x7fab7c1a0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab82600640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c10e6e0 0x7fab7c19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81200640 1 -- 172.21.15.37:0/1362051265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab7c100b30 msgr2=0x7fab7c1a0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81200640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab7c100b30 0x7fab7c1a0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81200640 1 -- 172.21.15.37:0/1362051265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c10e6e0 msgr2=0x7fab7c19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81200640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c10e6e0 0x7fab7c19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81200640 1 -- 172.21.15.37:0/1362051265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c19ad90 con 0x7fab7c1014e0 2024-04-26T02:02:50.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.446+0000 7fab81200640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 0x7fab7c1a09a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab68005ab0 tx=0x7fab68003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.447+0000 7fab7a000640 1 -- 172.21.15.37:0/1362051265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6803f070 con 0x7fab7c1014e0 2024-04-26T02:02:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.447+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c19b020 con 0x7fab7c1014e0 2024-04-26T02:02:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.447+0000 7fab7a000640 1 -- 172.21.15.37:0/1362051265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab68031d60 con 0x7fab7c1014e0 2024-04-26T02:02:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.447+0000 7fab7a000640 1 -- 172.21.15.37:0/1362051265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68035030 con 0x7fab7c1014e0 2024-04-26T02:02:50.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.447+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c1a7320 con 0x7fab7c1014e0 2024-04-26T02:02:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 Processor -- start 2024-04-26T02:02:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab7a000640 1 -- 172.21.15.37:0/1362051265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fab68042070 con 0x7fab7c1014e0 2024-04-26T02:02:50.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 -- start start 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e09b0 0x555baa2e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 0x555baa2eb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2edf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555baa246900 con 0x555baa2e12d0 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555baa1ecef0 con 0x555baa2ebb90 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.451+0000 7fab87410740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555baa2428c0 con 0x555baa2e09b0 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 0x555baa2eb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81200640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2edf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 0x555baa2eb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43536/0 (socket says 172.21.15.37:43536) 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 -- 172.21.15.37:0/738621921 learned_addr learned my addr 172.21.15.37:0/738621921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab82600640 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e09b0 0x555baa2e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 -- 172.21.15.37:0/738621921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e09b0 msgr2=0x555baa2e0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e09b0 0x555baa2e0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 -- 172.21.15.37:0/738621921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 msgr2=0x555baa2edf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2edf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab81c00640 1 -- 172.21.15.37:0/738621921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba9f59210 con 0x555baa2e12d0 2024-04-26T02:02:50.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.452+0000 7fab82600640 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e09b0 0x555baa2e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab81c00640 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 0x555baa2eb650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab7000bcd0 tx=0x7fab700107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b437aaa8946e5441 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab6ea00640 1 -- 172.21.15.37:0/738621921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70005220 con 0x555baa2e12d0 2024-04-26T02:02:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab7a000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab640a3d00 0x7fab640a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab6ea00640 1 -- 172.21.15.37:0/738621921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab70004c50 con 0x555baa2e12d0 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab6ea00640 1 -- 172.21.15.37:0/738621921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab70011680 con 0x555baa2e12d0 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab7a000640 1 -- 172.21.15.37:0/1362051265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab680f0f80 con 0x7fab7c1014e0 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.453+0000 7fab82600640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab640a3d00 0x7fab640a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 -- 172.21.15.37:0/738621921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 msgr2=0x555baa2eb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab82600640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab640a3d00 0x7fab640a61c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab74009fd0 tx=0x7fab74009500 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 0x555baa2eb650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fab7000bcd0 tx=0x7fab700107a0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 -- 172.21.15.37:0/738621921 shutdown_connections 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2edf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e12d0 0x555baa2eb650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 --2- 172.21.15.37:0/738621921 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e09b0 0x555baa2e0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 -- 172.21.15.37:0/738621921 >> 172.21.15.37:0/738621921 conn(0x555baa2dcec0 msgr2=0x555baa2dd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 -- 172.21.15.37:0/738621921 shutdown_connections 2024-04-26T02:02:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.454+0000 7fab87410740 1 -- 172.21.15.37:0/738621921 wait complete. 2024-04-26T02:02:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.455+0000 7fab87410740 1 -- start start 2024-04-26T02:02:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.455+0000 7fab87410740 1 Processor -- start 2024-04-26T02:02:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2e8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2e9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2e95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555baa2ee7c0 con 0x555baa2e09b0 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x555baa200310 con 0x555baa2ebb90 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x555baa2ee940 con 0x555baa2e12d0 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2e9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2e8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab87410740 10 client.0 _ll_register_callbacks cb 0x555baa1cecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2e9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35036/0 (socket says 172.21.15.37:35036) 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 -- 172.21.15.37:0/565732127 learned_addr learned my addr 172.21.15.37:0/565732127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab82600640 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2e95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 -- 172.21.15.37:0/565732127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 msgr2=0x555baa2e95d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2e95d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 -- 172.21.15.37:0/565732127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 msgr2=0x555baa2e8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2e8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.456+0000 7fab81200640 1 -- 172.21.15.37:0/565732127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baa025620 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab81200640 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2e9090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab68005ec0 tx=0x7fab6803b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da16c51fb177cb72 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab63400640 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab63400640 1 -- 172.21.15.37:0/565732127 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68049070 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab63400640 1 -- 172.21.15.37:0/565732127 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab6803b4f0 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab70004c50 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab63400640 1 -- 172.21.15.37:0/565732127 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fab680bcd70 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab87410740 10 client.25651 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.457+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555baa2ee7c0 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab63400640 1 -- 172.21.15.37:0/565732127 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab70004c50 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab63400640 1 -- 172.21.15.37:0/565732127 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fab680ee9b0 con 0x555baa2e12d0 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab87410740 10 client.25651 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab87410740 10 client.25651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab87410740 20 client.25651 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab87410740 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa2fbea0 0x555baa2fe310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab87410740 4 client.25651 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555baa25a8b0 con 0x555baa2fbea0 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.458+0000 7fab81c00640 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa2fbea0 0x555baa2fe310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.459+0000 7fab81c00640 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa2fbea0 0x555baa2fe310 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8550fb6cfa500c3 server_cookie=1d73f33ad8fbdfeb in_seq=0 out_seq=0 2024-04-26T02:02:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.459+0000 7fab63400640 10 client.25651 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:50.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:49 smithi037 ceph-mon[27906]: pgmap v149: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 9.2 KiB/s wr, 219 op/s 2024-04-26T02:02:50.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:49 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2730526385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:02:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab63400640 1 -- 172.21.15.37:0/565732127 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555baa25a8b0 con 0x555baa2fbea0 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab63400640 10 client.25651 handle_command_reply: tid=0 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 client.25651 shutdown 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 msgr2=0x555baa2e9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2e9090 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fab68005ec0 tx=0x7fab6803b460 comp rx=0 tx=0).stop 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 shutdown_connections 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa2fbea0 0x555baa2fe310 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa2e95d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2e9090 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 --2- 172.21.15.37:0/565732127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2e8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab63400640 0 client.25651 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 >> 172.21.15.37:0/565732127 conn(0x555baa2dcec0 msgr2=0x555baa2e2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 shutdown_connections 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 1 -- 172.21.15.37:0/565732127 wait complete. 2024-04-26T02:02:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.460+0000 7fab87410740 20 client.25651 trim_cache size 0 max 16384 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 Processor -- start 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 -- start start 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2e09b0 0x555baa2f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2ebb90 0x555baa303c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1adc30 con 0x555baa2ebb90 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b04b0 con 0x555baa2e09b0 2024-04-26T02:02:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.477+0000 7fab87410740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1ae2a0 con 0x555baa2e12d0 2024-04-26T02:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2e09b0 0x555baa2f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab82600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2ebb90 0x555baa303c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2dc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35040/0 (socket says 172.21.15.37:35040) 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 -- 172.21.15.37:0/3311126093 learned_addr learned my addr 172.21.15.37:0/3311126093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81c00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2e09b0 0x555baa2f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:40772/0 (socket says 172.21.15.37:40772) 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 -- 172.21.15.37:0/3311126093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2e09b0 msgr2=0x555baa2f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2e09b0 0x555baa2f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 -- 172.21.15.37:0/3311126093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2ebb90 msgr2=0x555baa303c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2ebb90 0x555baa303c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 -- 172.21.15.37:0/3311126093 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555baa2e4740 con 0x555baa2e12d0 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab82600640 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2ebb90 0x555baa303c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.478+0000 7fab81200640 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2dc7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab680bcf30 tx=0x7fab68009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=204a2fdc4bf00e03 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.479+0000 7fab6e000640 1 -- 172.21.15.37:0/3311126093 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68049040 con 0x555baa2e12d0 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.479+0000 7fab6e000640 1 -- 172.21.15.37:0/3311126093 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab680c12a0 con 0x555baa2e12d0 2024-04-26T02:02:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.479+0000 7fab6e000640 1 -- 172.21.15.37:0/3311126093 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680bc900 con 0x555baa2e12d0 2024-04-26T02:02:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 -- 172.21.15.37:0/3311126093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 msgr2=0x555baa2dc7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2dc7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fab680bcf30 tx=0x7fab68009260 comp rx=0 tx=0).stop 2024-04-26T02:02:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 -- 172.21.15.37:0/3311126093 shutdown_connections 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2ebb90 0x555baa303c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2dc7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 --2- 172.21.15.37:0/3311126093 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2e09b0 0x555baa2f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 -- 172.21.15.37:0/3311126093 >> 172.21.15.37:0/3311126093 conn(0x555baa2dcec0 msgr2=0x555baa2dd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 -- 172.21.15.37:0/3311126093 shutdown_connections 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 -- 172.21.15.37:0/3311126093 wait complete. 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.480+0000 7fab87410740 1 -- start start 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 Processor -- start 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2f0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2f13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa35a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1ae970 con 0x555baa2e09b0 2024-04-26T02:02:50.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1afde0 con 0x555baa2ebb90 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1af040 con 0x555baa2e12d0 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab87410740 10 client.0 _ll_register_callbacks cb 0x7fab68003e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.481+0000 7fab81c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2f0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2f13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab81200640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2f13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35054/0 (socket says 172.21.15.37:35054) 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab81c00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2f0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43574/0 (socket says 172.21.15.37:43574) 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab81200640 1 -- 172.21.15.37:0/2186112610 learned_addr learned my addr 172.21.15.37:0/2186112610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab82600640 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa35a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab82600640 1 -- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 msgr2=0x555baa2f13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab82600640 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2f13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab82600640 1 -- 172.21.15.37:0/2186112610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 msgr2=0x555baa2f0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab82600640 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2f0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.482 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.482+0000 7fab82600640 1 -- 172.21.15.37:0/2186112610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555baa35aa90 con 0x555baa2ebb90 2024-04-26T02:02:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab82600640 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa35a4d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab7c06a850 tx=0x7fab74004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f971b9af4d0bb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab63400640 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:02:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab63400640 1 -- 172.21.15.37:0/2186112610 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7402d060 con 0x555baa2ebb90 2024-04-26T02:02:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab63400640 1 -- 172.21.15.37:0/2186112610 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fab74010640 con 0x555baa2ebb90 2024-04-26T02:02:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab680c12a0 con 0x555baa2ebb90 2024-04-26T02:02:50.484 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab63400640 1 -- 172.21.15.37:0/2186112610 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fab74010840 con 0x555baa2ebb90 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab87410740 10 client.25592 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.483+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x555baa361eb0 con 0x555baa2ebb90 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab63400640 1 -- 172.21.15.37:0/2186112610 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab680c12a0 con 0x555baa2ebb90 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab63400640 1 -- 172.21.15.37:0/2186112610 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fab74028a70 con 0x555baa2ebb90 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab87410740 10 client.25592 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab87410740 10 client.25592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab87410740 20 client.25592 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab87410740 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa362600 0x555baa364a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab87410740 4 client.25592 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555baa254320 con 0x555baa362600 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.484+0000 7fab81c00640 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa362600 0x555baa364a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.485+0000 7fab81c00640 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa362600 0x555baa364a20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7c52d9025533907 server_cookie=647ee1647a0543e1 in_seq=0 out_seq=0 2024-04-26T02:02:50.485 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.485+0000 7fab63400640 10 client.25592 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab63400640 1 -- 172.21.15.37:0/2186112610 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555baa254320 con 0x555baa362600 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab63400640 10 client.25592 handle_command_reply: tid=0 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 client.25592 shutdown 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 msgr2=0x555baa35a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa35a4d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fab7c06a850 tx=0x7fab74004a40 comp rx=0 tx=0).stop 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 shutdown_connections 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x555baa362600 0x555baa364a20 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x555baa2ebb90 0x555baa35a4d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x555baa2e12d0 0x555baa2f13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 --2- 172.21.15.37:0/2186112610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555baa2e09b0 0x555baa2f0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab63400640 0 client.25592 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 >> 172.21.15.37:0/2186112610 conn(0x555baa2dcec0 msgr2=0x555baa2e73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 shutdown_connections 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 1 -- 172.21.15.37:0/2186112610 wait complete. 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.496+0000 7fab87410740 20 client.25592 trim_cache size 0 max 16384 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-04-26T02:02:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab640a3d00 msgr2=0x7fab640a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab640a3d00 0x7fab640a61c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fab74009fd0 tx=0x7fab74009500 comp rx=0 tx=0).stop 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 msgr2=0x7fab7c1a09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 0x7fab7c1a09a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab68005ab0 tx=0x7fab68003c80 comp rx=0 tx=0).stop 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 shutdown_connections 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fab640a3d00 0x7fab640a61c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fab7c10e6e0 0x7fab7c19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fab7c1014e0 0x7fab7c1a09a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 --2- 172.21.15.37:0/1362051265 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fab7c100b30 0x7fab7c1a0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 >> 172.21.15.37:0/1362051265 conn(0x7fab7c0fc730 msgr2=0x7fab7c0feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 shutdown_connections 2024-04-26T02:02:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-04-26T02:02:50.497+0000 7fab83000640 1 -- 172.21.15.37:0/1362051265 wait complete. 2024-04-26T02:02:50.631 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:02:52.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:52 smithi045 ceph-mon[32520]: pgmap v150: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 216 op/s 2024-04-26T02:02:52.390 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:52 smithi037 ceph-mon[27906]: pgmap v150: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 216 op/s 2024-04-26T02:02:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:52 smithi071 ceph-mon[32578]: pgmap v150: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 6.7 KiB/s wr, 216 op/s 2024-04-26T02:02:52.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 -- 172.21.15.37:0/3503186204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 msgr2=0x7fa0141096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 --2- 172.21.15.37:0/3503186204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141096b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa00400b0d0 tx=0x7fa00400b3e0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 -- 172.21.15.37:0/3503186204 shutdown_connections 2024-04-26T02:02:52.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 --2- 172.21.15.37:0/3503186204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141096b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 --2- 172.21.15.37:0/3503186204 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa01410e3b0 0x7fa014115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 --2- 172.21.15.37:0/3503186204 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 0x7fa01410dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 -- 172.21.15.37:0/3503186204 >> 172.21.15.37:0/3503186204 conn(0x7fa014076210 msgr2=0x7fa014107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 -- 172.21.15.37:0/3503186204 shutdown_connections 2024-04-26T02:02:52.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.989+0000 7fa01a000700 1 -- 172.21.15.37:0/3503186204 wait complete. 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.990+0000 7fa01a000700 1 Processor -- start 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 -- start start 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 0x7fa0141a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa01410e3b0 0x7fa01419f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa014119290 con 0x7fa0141092d0 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa014119110 con 0x7fa014109c80 2024-04-26T02:02:52.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa01a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa014119410 con 0x7fa01410e3b0 2024-04-26T02:02:52.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.991+0000 7fa013400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 0x7fa0141a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa013400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141a53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43598/0 (socket says 172.21.15.37:43598) 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa013400700 1 -- 172.21.15.37:0/200528038 learned_addr learned my addr 172.21.15.37:0/200528038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa013e00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa01410e3b0 0x7fa01419f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 -- 172.21.15.37:0/200528038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa01410e3b0 msgr2=0x7fa01419f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa01410e3b0 0x7fa01419f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 -- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 msgr2=0x7fa0141a53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141a53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 -- 172.21.15.37:0/200528038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa01419fd10 con 0x7fa014109c80 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.992+0000 7fa00aa00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 0x7fa0141a5930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00e9c0 tx=0x7f9ffc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.993+0000 7fa010c00700 1 -- 172.21.15.37:0/200528038 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc00cb70 con 0x7fa014109c80 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.993+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0141a0000 con 0x7fa014109c80 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.993+0000 7fa010c00700 1 -- 172.21.15.37:0/200528038 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ffc00ee60 con 0x7fa014109c80 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.993+0000 7fa010c00700 1 -- 172.21.15.37:0/200528038 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc003d60 con 0x7fa014109c80 2024-04-26T02:02:52.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.993+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0141ac2c0 con 0x7fa014109c80 2024-04-26T02:02:52.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.996+0000 7fa010c00700 1 -- 172.21.15.37:0/200528038 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9ffc00cd10 con 0x7fa014109c80 2024-04-26T02:02:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.997+0000 7fa024a4cb80 1 Processor -- start 2024-04-26T02:02:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 -- start start 2024-04-26T02:02:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa010c00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9ff80a3db0 0x7f9ff80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62e890 0x55c2de62ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 0x55c2de639b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de63c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c2de581f00 con 0x55c2de62e890 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c2de5287a0 con 0x55c2de62f1b0 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c2de57dec0 con 0x55c2de63a060 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa010c00700 1 -- 172.21.15.37:0/200528038 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9ffc0d0970 con 0x7fa014109c80 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.998+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 0x55c2de639b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 0x55c2de639b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:40814/0 (socket says 172.21.15.37:40814) 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 -- 172.21.15.37:0/1623978241 learned_addr learned my addr 172.21.15.37:0/1623978241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa013400700 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62e890 0x55c2de62ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa013e00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9ff80a3db0 0x7f9ff80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 -- 172.21.15.37:0/1623978241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 msgr2=0x55c2de63c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de63c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 -- 172.21.15.37:0/1623978241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62e890 msgr2=0x55c2de62ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62e890 0x55c2de62ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 -- 172.21.15.37:0/1623978241 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2de51e6f0 con 0x55c2de62f1b0 2024-04-26T02:02:52.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:52.999+0000 7fa00aa00700 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 0x55c2de639b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000c00 tx=0x7f9ffc09c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d5a1e8649abc062c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.000+0000 7fa008c00700 1 -- 172.21.15.37:0/1623978241 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc04ab90 con 0x55c2de62f1b0 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.000+0000 7fa008c00700 1 -- 172.21.15.37:0/1623978241 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ffc09cfe0 con 0x55c2de62f1b0 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.000+0000 7fa008c00700 1 -- 172.21.15.37:0/1623978241 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc049f20 con 0x55c2de62f1b0 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.000+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1623978241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 msgr2=0x55c2de639b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.000+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 0x55c2de639b20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc000c00 tx=0x7f9ffc09c4a0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.000+0000 7fa013e00700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9ff80a3db0 0x7f9ff80a6270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0097a0 tx=0x7fa00c0099d0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1623978241 shutdown_connections 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62e890 0x55c2de62ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de63c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1623978241 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62f1b0 0x55c2de639b20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1623978241 >> 172.21.15.37:0/1623978241 conn(0x55c2de55d610 msgr2=0x55c2de62ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1623978241 shutdown_connections 2024-04-26T02:02:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1623978241 wait complete. 2024-04-26T02:02:53.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.001+0000 7fa024a4cb80 1 -- start start 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 Processor -- start 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62e890 0x55c2de639770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de636a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 0x55c2de636fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c2de63cb80 con 0x55c2de62f1b0 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x55c2de53b8d0 con 0x55c2de63a060 2024-04-26T02:02:53.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x55c2de63cd00 con 0x55c2de62e890 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.002+0000 7fa024a4cb80 10 client.0 _ll_register_callbacks cb 0x55c2de51a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de636a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 0x55c2de636fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 0x55c2de636fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:40816/0 (socket says 172.21.15.37:40816) 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 -- 172.21.15.37:0/2326874559 learned_addr learned my addr 172.21.15.37:0/2326874559 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de636a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43608/0 (socket says 172.21.15.37:43608) 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013400700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62e890 0x55c2de639770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 -- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62e890 msgr2=0x55c2de639770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62e890 0x55c2de639770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 -- 172.21.15.37:0/2326874559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 msgr2=0x55c2de636a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de636a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa013e00700 1 -- 172.21.15.37:0/2326874559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2de4cf940 con 0x55c2de63a060 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.003+0000 7fa00aa00700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de636a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.004+0000 7fa013e00700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 0x55c2de636fb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0092a0 tx=0x7fa00c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c966dcf9daf483b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:53.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.004+0000 7fa001600700 10 client.0 ms_handle_connect on v2:172.21.15.45:3300/0 2024-04-26T02:02:53.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.006+0000 7fa001600700 1 -- 172.21.15.37:0/2326874559 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00c01c610 con 0x55c2de63a060 2024-04-26T02:02:53.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.006+0000 7fa001600700 1 -- 172.21.15.37:0/2326874559 <== mon.2 v2:172.21.15.45:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7fa00c01c7b0 con 0x55c2de63a060 2024-04-26T02:02:53.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.006+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ffc09cfe0 con 0x55c2de63a060 2024-04-26T02:02:53.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.006+0000 7fa001600700 1 -- 172.21.15.37:0/2326874559 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7fa00c01c9b0 con 0x55c2de63a060 2024-04-26T02:02:53.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.006+0000 7fa024a4cb80 10 client.25604 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:53.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.006+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c2de518b60 con 0x55c2de63a060 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa001600700 1 -- 172.21.15.37:0/2326874559 <== mon.2 v2:172.21.15.45:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ffc09cfe0 con 0x55c2de63a060 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa001600700 1 -- 172.21.15.37:0/2326874559 <== mon.2 v2:172.21.15.45:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7fa00c019030 con 0x55c2de63a060 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa024a4cb80 10 client.25604 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa024a4cb80 10 client.25604 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa024a4cb80 20 client.25604 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6497e0 0x55c2de64bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa024a4cb80 4 client.25604 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.007+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2de58f6f0 con 0x55c2de6497e0 2024-04-26T02:02:53.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.008+0000 7fa013400700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6497e0 0x55c2de64bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.008+0000 7fa013400700 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6497e0 0x55c2de64bbd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f55ef7f50659d9a5 server_cookie=3b8ea276345e5a25 in_seq=0 out_seq=0 2024-04-26T02:02:53.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.008+0000 7fa001600700 10 client.25604 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:53.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.009+0000 7fa001600700 1 -- 172.21.15.37:0/2326874559 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2de58f6f0 con 0x55c2de6497e0 2024-04-26T02:02:53.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.009+0000 7fa001600700 10 client.25604 handle_command_reply: tid=0 2024-04-26T02:02:53.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.009+0000 7fa024a4cb80 1 client.25604 shutdown 2024-04-26T02:02:53.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.009+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 msgr2=0x55c2de636fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.009+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 0x55c2de636fb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0092a0 tx=0x7fa00c0047c0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 shutdown_connections 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6497e0 0x55c2de64bbd0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de63a060 0x55c2de636fb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62e890 0x55c2de639770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/2326874559 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de636a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa001600700 0 client.25604 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 >> 172.21.15.37:0/2326874559 conn(0x55c2de55d610 msgr2=0x55c2de62a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 shutdown_connections 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 1 -- 172.21.15.37:0/2326874559 wait complete. 2024-04-26T02:02:53.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.010+0000 7fa024a4cb80 20 client.25604 trim_cache size 0 max 16384 2024-04-26T02:02:53.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.031+0000 7fa024a4cb80 1 Processor -- start 2024-04-26T02:02:53.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 -- start start 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de6359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de645a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de633cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b18f0 con 0x55c2de62f1b0 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b4170 con 0x55c2de62e890 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b1f60 con 0x55c2de63a060 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de645a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de645a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43612/0 (socket says 172.21.15.37:43612) 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa00aa00700 1 -- 172.21.15.37:0/1275186297 learned_addr learned my addr 172.21.15.37:0/1275186297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa013400700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de6359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.032+0000 7fa013e00700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de633cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013400700 1 -- 172.21.15.37:0/1275186297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 msgr2=0x55c2de633cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013400700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de633cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013400700 1 -- 172.21.15.37:0/1275186297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 msgr2=0x55c2de645a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013400700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de645a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013400700 1 -- 172.21.15.37:0/1275186297 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2de6393b0 con 0x55c2de62e890 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013e00700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de633cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa00aa00700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de645a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa013400700 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de6359d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55c2de63cde0 tx=0x7fa00400b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cf5f6ca7c63b7dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa003e00700 1 -- 172.21.15.37:0/1275186297 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa00400bde0 con 0x55c2de62e890 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa003e00700 1 -- 172.21.15.37:0/1275186297 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa004029080 con 0x55c2de62e890 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.033+0000 7fa003e00700 1 -- 172.21.15.37:0/1275186297 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa004007700 con 0x55c2de62e890 2024-04-26T02:02:53.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1275186297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 msgr2=0x55c2de6359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de6359d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x55c2de63cde0 tx=0x7fa00400b3e0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1275186297 shutdown_connections 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de63a060 0x55c2de633cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de6359d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/1275186297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de62f1b0 0x55c2de645a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1275186297 >> 172.21.15.37:0/1275186297 conn(0x55c2de55d610 msgr2=0x55c2de62a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1275186297 shutdown_connections 2024-04-26T02:02:53.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.034+0000 7fa024a4cb80 1 -- 172.21.15.37:0/1275186297 wait complete. 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 -- start start 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 Processor -- start 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de63eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 0x55c2de63f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de63a060 0x55c2de63f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b2630 con 0x55c2de63a060 2024-04-26T02:02:53.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b3aa0 con 0x55c2de62e890 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.035+0000 7fa024a4cb80 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fa0141b2d00 con 0x55c2de62f1b0 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa024a4cb80 10 client.0 _ll_register_callbacks cb 0x55c2de51a790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 0x55c2de63f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 0x55c2de63f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:35102/0 (socket says 172.21.15.37:35102) 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 -- 172.21.15.37:0/3155290209 learned_addr learned my addr 172.21.15.37:0/3155290209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa013400700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de63eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa013e00700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de63a060 0x55c2de63f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 -- 172.21.15.37:0/3155290209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 msgr2=0x55c2de63eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de63eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 -- 172.21.15.37:0/3155290209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de63a060 msgr2=0x55c2de63f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de63a060 0x55c2de63f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 -- 172.21.15.37:0/3155290209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2de6afec0 con 0x55c2de62f1b0 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa013400700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de63eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:53.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa013e00700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de63a060 0x55c2de63f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa00aa00700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 0x55c2de63f0d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc09c3b0 tx=0x7f9ffc0a0d40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14a62c7e00cdad41 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.036+0000 7fa001600700 10 client.0 ms_handle_connect on v2:172.21.15.71:3300/0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa001600700 1 -- 172.21.15.37:0/3155290209 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0ce510 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa001600700 1 -- 172.21.15.37:0/3155290209 <== mon.1 v2:172.21.15.71:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8469+0+0 (secure 0 0 0) 0x7f9ffc09cb40 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa004029080 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa001600700 1 -- 172.21.15.37:0/3155290209 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f9ffc09aca0 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 10 client.25672 fetch_fsmap learned FSMap version 19 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=4+}) -- 0x55c2de6b1180 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa001600700 1 -- 172.21.15.37:0/3155290209 <== mon.1 v2:172.21.15.71:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa004029080 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa001600700 1 -- 172.21.15.37:0/3155290209 <== mon.1 v2:172.21.15.71:3300/0 5 ==== fsmap(e 19) ==== 8322+0+0 (secure 0 0 0) 0x7f9ffc098040 con 0x55c2de62f1b0 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 10 client.25672 fetch_fsmap finished waiting for FSMap version 19 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 10 client.25672 resolve_mds: validated gid 24514 aka daemon mds.i 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 20 client.25672 populate_metadata read hostname 'smithi037' 2024-04-26T02:02:53.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.037+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6b1410 0x55c2de6b3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:02:53.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.038+0000 7fa024a4cb80 4 client.25672 mds_command: new command op to 24514 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-26T02:02:53.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.038+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 --> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55c2de589790 con 0x55c2de6b1410 2024-04-26T02:02:53.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.038+0000 7fa013400700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6b1410 0x55c2de6b3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.038+0000 7fa013400700 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6b1410 0x55c2de6b3800 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d75f8a9a28eb6c server_cookie=f17cf176bdb17b3e in_seq=0 out_seq=0 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.038+0000 7fa001600700 10 client.25672 ms_handle_connect on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa001600700 1 -- 172.21.15.37:0/3155290209 <== mds.0 v2:172.21.15.71:6838/1148187051 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55c2de589790 con 0x55c2de6b1410 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa001600700 10 client.25672 handle_command_reply: tid=0 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 client.25672 shutdown 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 msgr2=0x55c2de63f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 0x55c2de63f0d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc09c3b0 tx=0x7f9ffc0a0d40 comp rx=0 tx=0).stop 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 shutdown_connections 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:6838/1148187051,v1:172.21.15.71:6839/1148187051] conn(0x55c2de6b1410 0x55c2de6b3800 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2de63a060 0x55c2de63f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x55c2de62e890 0x55c2de63eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 --2- 172.21.15.37:0/3155290209 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x55c2de62f1b0 0x55c2de63f0d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa001600700 0 client.25672 ms_handle_reset on v2:172.21.15.71:6838/1148187051 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 >> 172.21.15.37:0/3155290209 conn(0x55c2de55d610 msgr2=0x55c2de644a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 shutdown_connections 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 1 -- 172.21.15.37:0/3155290209 wait complete. 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.039+0000 7fa024a4cb80 20 client.25672 trim_cache size 0 max 16384 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 19, 2024-04-26T02:02:53.041 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24514, 2024-04-26T02:02:53.042 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-04-26T02:02:53.042 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": {} 2024-04-26T02:02:53.042 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:02:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9ff80a3db0 msgr2=0x7f9ff80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9ff80a3db0 0x7f9ff80a6270 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa00c0097a0 tx=0x7fa00c0099d0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 msgr2=0x7fa0141a5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 0x7fa0141a5930 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc00e9c0 tx=0x7f9ffc00ecd0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 shutdown_connections 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0141092d0 0x7fa0141a53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f9ff80a3db0 0x7f9ff80a6270 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fa01410e3b0 0x7fa01419f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 --2- 172.21.15.37:0/200528038 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fa014109c80 0x7fa0141a5930 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 >> 172.21.15.37:0/200528038 conn(0x7fa014076210 msgr2=0x7fa014077e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 shutdown_connections 2024-04-26T02:02:53.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:02:53.040+0000 7fa01a000700 1 -- 172.21.15.37:0/200528038 wait complete. 2024-04-26T02:02:53.917 INFO:tasks.quiescer:done joining Quiescers 2024-04-26T02:02:53.918 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T02:02:53.929 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T02:02:53.957 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T02:02:53.969 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T02:02:53.969 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T02:02:53.969 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T02:02:53.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T02:02:54.100 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T02:02:54.169 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:54 smithi037 ceph-mon[27906]: pgmap v151: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.1 KiB/s wr, 220 op/s 2024-04-26T02:02:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:54 smithi045 ceph-mon[32520]: pgmap v151: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.1 KiB/s wr, 220 op/s 2024-04-26T02:02:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:54 smithi071 ceph-mon[32578]: pgmap v151: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 9.1 KiB/s wr, 220 op/s 2024-04-26T02:02:56.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:56 smithi071 ceph-mon[32578]: pgmap v152: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 KiB/s wr, 219 op/s 2024-04-26T02:02:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:56 smithi037 ceph-mon[27906]: pgmap v152: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 KiB/s wr, 219 op/s 2024-04-26T02:02:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:56 smithi045 ceph-mon[32520]: pgmap v152: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 KiB/s wr, 219 op/s 2024-04-26T02:02:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:02:58 smithi071 ceph-mon[32578]: pgmap v153: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 KiB/s wr, 219 op/s 2024-04-26T02:02:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:02:58 smithi037 ceph-mon[27906]: pgmap v153: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 KiB/s wr, 219 op/s 2024-04-26T02:02:58.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:02:58 smithi045 ceph-mon[32520]: pgmap v153: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 6.9 KiB/s wr, 219 op/s 2024-04-26T02:03:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:00 smithi071 ceph-mon[32578]: pgmap v154: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.9 KiB/s wr, 220 op/s 2024-04-26T02:03:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:00 smithi037 ceph-mon[27906]: pgmap v154: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.9 KiB/s wr, 220 op/s 2024-04-26T02:03:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:00 smithi045 ceph-mon[32520]: pgmap v154: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 6.9 KiB/s wr, 220 op/s 2024-04-26T02:03:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:02 smithi071 ceph-mon[32578]: pgmap v155: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T02:03:02.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:02 smithi037 ceph-mon[27906]: pgmap v155: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T02:03:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:02 smithi045 ceph-mon[32520]: pgmap v155: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 2.5 KiB/s wr, 5 op/s 2024-04-26T02:03:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:04 smithi071 ceph-mon[32578]: pgmap v156: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-26T02:03:04.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:04 smithi037 ceph-mon[27906]: pgmap v156: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-26T02:03:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:04 smithi045 ceph-mon[32520]: pgmap v156: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 21 op/s 2024-04-26T02:03:06.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:05 smithi071 ceph-mon[32578]: pgmap v157: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:06.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:05 smithi037 ceph-mon[27906]: pgmap v157: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:05 smithi045 ceph-mon[32520]: pgmap v157: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:08.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:07 smithi071 ceph-mon[32578]: pgmap v158: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:03:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:07 smithi037 ceph-mon[27906]: pgmap v158: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:03:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:07 smithi045 ceph-mon[32520]: pgmap v158: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:03:10.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:09 smithi071 ceph-mon[32578]: pgmap v159: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:10.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:09 smithi037 ceph-mon[27906]: pgmap v159: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:09 smithi045 ceph-mon[32520]: pgmap v159: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:12.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:11 smithi071 ceph-mon[32578]: pgmap v160: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:03:12.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:11 smithi037 ceph-mon[27906]: pgmap v160: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:03:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:11 smithi045 ceph-mon[32520]: pgmap v160: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:03:13.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:12 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:13.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:12 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:13.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:12 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:03:14.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:13 smithi071 ceph-mon[32578]: pgmap v161: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:14.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:13 smithi037 ceph-mon[27906]: pgmap v161: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:13 smithi045 ceph-mon[32520]: pgmap v161: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:03:15.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:14 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:15.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:14 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:15.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:14 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:03:15.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:14 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:15.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:14 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:15.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:14 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:03:15.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:14 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:03:15.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:14 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:03:15.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:14 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:03:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:16 smithi045 ceph-mon[32520]: pgmap v162: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:03:16.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:16 smithi071 ceph-mon[32578]: pgmap v162: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:03:16.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:16 smithi037 ceph-mon[27906]: pgmap v162: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:03:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:18 smithi045 ceph-mon[32520]: pgmap v163: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:03:18.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:18 smithi037 ceph-mon[27906]: pgmap v163: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:03:18.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:18 smithi071 ceph-mon[32578]: pgmap v163: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:03:20.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:20 smithi045 ceph-mon[32520]: pgmap v164: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T02:03:20.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:20 smithi071 ceph-mon[32578]: pgmap v164: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T02:03:20.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:20 smithi037 ceph-mon[27906]: pgmap v164: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T02:03:22.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:22 smithi045 ceph-mon[32520]: pgmap v165: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:22.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:22 smithi071 ceph-mon[32578]: pgmap v165: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:22.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:22 smithi037 ceph-mon[27906]: pgmap v165: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:24 smithi045 ceph-mon[32520]: pgmap v166: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:24.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:24 smithi071 ceph-mon[32578]: pgmap v166: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:24.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:24 smithi037 ceph-mon[27906]: pgmap v166: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:26.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:26 smithi071 ceph-mon[32578]: pgmap v167: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:26.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:26 smithi037 ceph-mon[27906]: pgmap v167: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:26.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:26 smithi045 ceph-mon[32520]: pgmap v167: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:28.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:28 smithi071 ceph-mon[32578]: pgmap v168: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T02:03:28.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:28 smithi037 ceph-mon[27906]: pgmap v168: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T02:03:28.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:28 smithi045 ceph-mon[32520]: pgmap v168: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T02:03:30.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:30 smithi037 ceph-mon[27906]: pgmap v169: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:30.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:30 smithi071 ceph-mon[32578]: pgmap v169: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:30.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:30 smithi045 ceph-mon[32520]: pgmap v169: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:32.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:32 smithi071 ceph-mon[32578]: pgmap v170: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:32.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:32 smithi037 ceph-mon[27906]: pgmap v170: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:32.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:32 smithi045 ceph-mon[32520]: pgmap v170: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:34.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:34 smithi071 ceph-mon[32578]: pgmap v171: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:34.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:34 smithi037 ceph-mon[27906]: pgmap v171: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:34.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:34 smithi045 ceph-mon[32520]: pgmap v171: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:36.654 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:36 smithi071 ceph-mon[32578]: pgmap v172: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:36.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:36 smithi037 ceph-mon[27906]: pgmap v172: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:36.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:36 smithi045 ceph-mon[32520]: pgmap v172: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:38.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:38 smithi071 ceph-mon[32578]: pgmap v173: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:38.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:38 smithi037 ceph-mon[27906]: pgmap v173: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:38.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:38 smithi045 ceph-mon[32520]: pgmap v173: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:40.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:40 smithi071 ceph-mon[32578]: pgmap v174: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:40.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:40 smithi037 ceph-mon[27906]: pgmap v174: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:40.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:40 smithi045 ceph-mon[32520]: pgmap v174: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:42.557 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:42 smithi045 ceph-mon[32520]: pgmap v175: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:42.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:42 smithi071 ceph-mon[32578]: pgmap v175: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:42.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:42 smithi037 ceph-mon[27906]: pgmap v175: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:44.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:44 smithi071 ceph-mon[32578]: pgmap v176: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:44.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:44 smithi037 ceph-mon[27906]: pgmap v176: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:44.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:44 smithi045 ceph-mon[32520]: pgmap v176: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:46.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:46 smithi071 ceph-mon[32578]: pgmap v177: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:46.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:46 smithi037 ceph-mon[27906]: pgmap v177: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:46.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:46 smithi045 ceph-mon[32520]: pgmap v177: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:48.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:48 smithi071 ceph-mon[32578]: pgmap v178: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:48.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:48 smithi037 ceph-mon[27906]: pgmap v178: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:48.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:48 smithi045 ceph-mon[32520]: pgmap v178: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:50.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:50 smithi071 ceph-mon[32578]: pgmap v179: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:50.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:50 smithi037 ceph-mon[27906]: pgmap v179: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:50.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:50 smithi045 ceph-mon[32520]: pgmap v179: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:52.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:52 smithi071 ceph-mon[32578]: pgmap v180: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:52.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:52 smithi037 ceph-mon[27906]: pgmap v180: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:52.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:52 smithi045 ceph-mon[32520]: pgmap v180: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:53.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:53 smithi071 ceph-mon[32578]: pgmap v181: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:53.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:53 smithi037 ceph-mon[27906]: pgmap v181: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:53.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:53 smithi045 ceph-mon[32520]: pgmap v181: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:03:56.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:55 smithi071 ceph-mon[32578]: pgmap v182: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:56.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:55 smithi037 ceph-mon[27906]: pgmap v182: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:55 smithi045 ceph-mon[32520]: pgmap v182: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:58.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:57 smithi071 ceph-mon[32578]: pgmap v183: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:58.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:57 smithi037 ceph-mon[27906]: pgmap v183: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:03:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:57 smithi045 ceph-mon[32520]: pgmap v183: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:00.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:03:59 smithi071 ceph-mon[32578]: pgmap v184: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:00.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:03:59 smithi037 ceph-mon[27906]: pgmap v184: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:03:59 smithi045 ceph-mon[32520]: pgmap v184: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:02.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:01 smithi071 ceph-mon[32578]: pgmap v185: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:02.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:01 smithi037 ceph-mon[27906]: pgmap v185: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:01 smithi045 ceph-mon[32520]: pgmap v185: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:04.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:03 smithi037 ceph-mon[27906]: pgmap v186: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:04:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:03 smithi045 ceph-mon[32520]: pgmap v186: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:04:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:03 smithi071 ceph-mon[32578]: pgmap v186: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:04:06.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:05 smithi071 ceph-mon[32578]: pgmap v187: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:04:06.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:05 smithi037 ceph-mon[27906]: pgmap v187: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:04:06.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:05 smithi045 ceph-mon[32520]: pgmap v187: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:04:07.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:06 smithi037 ceph-mon[27906]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T02:04:07.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:06 smithi045 ceph-mon[32520]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T02:04:07.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:06 smithi071 ceph-mon[32578]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T02:04:08.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:07 smithi037 ceph-mon[27906]: pgmap v189: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T02:04:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:07 smithi045 ceph-mon[32520]: pgmap v189: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T02:04:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:07 smithi071 ceph-mon[32578]: pgmap v189: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T02:04:10.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:09 smithi037 ceph-mon[27906]: pgmap v190: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T02:04:10.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:09 smithi037 ceph-mon[27906]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T02:04:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:09 smithi045 ceph-mon[32520]: pgmap v190: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T02:04:10.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:09 smithi045 ceph-mon[32520]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T02:04:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:09 smithi071 ceph-mon[32578]: pgmap v190: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-04-26T02:04:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:09 smithi071 ceph-mon[32578]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T02:04:12.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:11 smithi037 ceph-mon[27906]: pgmap v192: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:04:12.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:11 smithi045 ceph-mon[32520]: pgmap v192: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:04:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:11 smithi071 ceph-mon[32578]: pgmap v192: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:04:14.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:13 smithi037 ceph-mon[27906]: pgmap v193: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:04:14.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:13 smithi045 ceph-mon[32520]: pgmap v193: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:04:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:13 smithi071 ceph-mon[32578]: pgmap v193: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:04:15.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:14 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:15.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:14 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:15.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:14 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:04:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:15 smithi045 ceph-mon[32520]: pgmap v194: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 230 B/s wr, 2 op/s 2024-04-26T02:04:16.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:15 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:16.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:15 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:16.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:15 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:04:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:15 smithi071 ceph-mon[32578]: pgmap v194: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 230 B/s wr, 2 op/s 2024-04-26T02:04:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:15 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:15 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:16.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:15 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:04:16.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:15 smithi037 ceph-mon[27906]: pgmap v194: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 230 B/s wr, 2 op/s 2024-04-26T02:04:16.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:15 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:04:16.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:15 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:04:16.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:15 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:04:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:17 smithi045 ceph-mon[32520]: pgmap v195: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:04:18.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:17 smithi037 ceph-mon[27906]: pgmap v195: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:04:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:17 smithi071 ceph-mon[32578]: pgmap v195: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:04:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:19 smithi045 ceph-mon[32520]: pgmap v196: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:04:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:19 smithi071 ceph-mon[32578]: pgmap v196: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:04:20.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:19 smithi037 ceph-mon[27906]: pgmap v196: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:04:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:21 smithi045 ceph-mon[32520]: pgmap v197: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 92 B/s wr, 1 op/s 2024-04-26T02:04:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:21 smithi071 ceph-mon[32578]: pgmap v197: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 92 B/s wr, 1 op/s 2024-04-26T02:04:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:21 smithi037 ceph-mon[27906]: pgmap v197: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 B/s rd, 92 B/s wr, 1 op/s 2024-04-26T02:04:24.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:23 smithi045 ceph-mon[32520]: pgmap v198: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:04:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:23 smithi071 ceph-mon[32578]: pgmap v198: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:04:24.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:23 smithi037 ceph-mon[27906]: pgmap v198: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:04:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:25 smithi071 ceph-mon[32578]: pgmap v199: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:25 smithi037 ceph-mon[27906]: pgmap v199: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:26.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:25 smithi045 ceph-mon[32520]: pgmap v199: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:27 smithi071 ceph-mon[32578]: pgmap v200: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:27 smithi037 ceph-mon[27906]: pgmap v200: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:27 smithi045 ceph-mon[32520]: pgmap v200: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:29 smithi071 ceph-mon[32578]: pgmap v201: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:29 smithi037 ceph-mon[27906]: pgmap v201: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:29 smithi045 ceph-mon[32520]: pgmap v201: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:32.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:31 smithi037 ceph-mon[27906]: pgmap v202: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:32.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:31 smithi045 ceph-mon[32520]: pgmap v202: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:31 smithi071 ceph-mon[32578]: pgmap v202: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:34.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:33 smithi037 ceph-mon[27906]: pgmap v203: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:33 smithi045 ceph-mon[32520]: pgmap v203: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:33 smithi071 ceph-mon[32578]: pgmap v203: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:36.205 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:35 smithi071 ceph-mon[32578]: pgmap v204: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:36.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:35 smithi037 ceph-mon[27906]: pgmap v204: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:35 smithi045 ceph-mon[32520]: pgmap v204: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:38.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:37 smithi037 ceph-mon[27906]: pgmap v205: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:37 smithi045 ceph-mon[32520]: pgmap v205: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:37 smithi071 ceph-mon[32578]: pgmap v205: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:40.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:39 smithi037 ceph-mon[27906]: pgmap v206: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:40.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:39 smithi045 ceph-mon[32520]: pgmap v206: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:39 smithi071 ceph-mon[32578]: pgmap v206: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:42.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:41 smithi037 ceph-mon[27906]: pgmap v207: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:41 smithi045 ceph-mon[32520]: pgmap v207: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:42.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:41 smithi071 ceph-mon[32578]: pgmap v207: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:44.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:43 smithi037 ceph-mon[27906]: pgmap v208: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:44.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:43 smithi045 ceph-mon[32520]: pgmap v208: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:43 smithi071 ceph-mon[32578]: pgmap v208: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:46.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:45 smithi037 ceph-mon[27906]: pgmap v209: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:45 smithi045 ceph-mon[32520]: pgmap v209: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:45 smithi071 ceph-mon[32578]: pgmap v209: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:48.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:47 smithi037 ceph-mon[27906]: pgmap v210: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:47 smithi045 ceph-mon[32520]: pgmap v210: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:47 smithi071 ceph-mon[32578]: pgmap v210: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:50.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:49 smithi037 ceph-mon[27906]: pgmap v211: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:50.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:49 smithi045 ceph-mon[32520]: pgmap v211: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:49 smithi071 ceph-mon[32578]: pgmap v211: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:52.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:51 smithi045 ceph-mon[32520]: pgmap v212: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:51 smithi071 ceph-mon[32578]: pgmap v212: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:51 smithi037 ceph-mon[27906]: pgmap v212: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:54.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:53 smithi045 ceph-mon[32520]: pgmap v213: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:53 smithi071 ceph-mon[32578]: pgmap v213: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:54.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:53 smithi037 ceph-mon[27906]: pgmap v213: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:04:56.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:55 smithi045 ceph-mon[32520]: pgmap v214: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:55 smithi071 ceph-mon[32578]: pgmap v214: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:56.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:55 smithi037 ceph-mon[27906]: pgmap v214: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:58.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:57 smithi045 ceph-mon[32520]: pgmap v215: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:57 smithi071 ceph-mon[32578]: pgmap v215: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:04:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:57 smithi037 ceph-mon[27906]: pgmap v215: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:00.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:04:59 smithi045 ceph-mon[32520]: pgmap v216: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:04:59 smithi071 ceph-mon[32578]: pgmap v216: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:04:59 smithi037 ceph-mon[27906]: pgmap v216: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:02.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:01 smithi045 ceph-mon[32520]: pgmap v217: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:01 smithi071 ceph-mon[32578]: pgmap v217: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:02.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:01 smithi037 ceph-mon[27906]: pgmap v217: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:04.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:03 smithi045 ceph-mon[32520]: pgmap v218: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:03 smithi071 ceph-mon[32578]: pgmap v218: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:03 smithi037 ceph-mon[27906]: pgmap v218: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:06.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:05 smithi045 ceph-mon[32520]: pgmap v219: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:05 smithi071 ceph-mon[32578]: pgmap v219: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:05 smithi037 ceph-mon[27906]: pgmap v219: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:08.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:08 smithi045 ceph-mon[32520]: pgmap v220: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:08 smithi071 ceph-mon[32578]: pgmap v220: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:08.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:08 smithi037 ceph-mon[27906]: pgmap v220: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:10 smithi071 ceph-mon[32578]: pgmap v221: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:10 smithi037 ceph-mon[27906]: pgmap v221: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:10 smithi045 ceph-mon[32520]: pgmap v221: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:12.307 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:12 smithi045 ceph-mon[32520]: pgmap v222: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:12 smithi071 ceph-mon[32578]: pgmap v222: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:12.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:12 smithi037 ceph-mon[27906]: pgmap v222: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-04-26T02:05:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:14 smithi071 ceph-mon[32578]: pgmap v223: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:14.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:14 smithi037 ceph-mon[27906]: pgmap v223: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:14 smithi045 ceph-mon[32520]: pgmap v223: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-04-26T02:05:16.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:16 smithi071 ceph-mon[32578]: pgmap v224: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:05:16.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:16 smithi037 ceph-mon[27906]: pgmap v224: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:05:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:16 smithi045 ceph-mon[32520]: pgmap v224: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:05:17.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:17 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:17.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:17 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:17.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:17 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:05:18.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:18 smithi037 ceph-mon[27906]: pgmap v225: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:05:18.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:18 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:18.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:18 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:18.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:18 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:05:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:18 smithi071 ceph-mon[32578]: pgmap v225: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:05:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:18 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:18 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:18 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:05:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:18 smithi045 ceph-mon[32520]: pgmap v225: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:05:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:18 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:05:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:18 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:05:18.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:18 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:05:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:20 smithi071 ceph-mon[32578]: pgmap v226: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:05:20.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:20 smithi037 ceph-mon[27906]: pgmap v226: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:05:20.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:20 smithi045 ceph-mon[32520]: pgmap v226: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:05:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:22 smithi071 ceph-mon[32578]: pgmap v227: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:22 smithi037 ceph-mon[27906]: pgmap v227: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:22.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:22 smithi045 ceph-mon[32520]: pgmap v227: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:24 smithi071 ceph-mon[32578]: pgmap v228: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:24 smithi037 ceph-mon[27906]: pgmap v228: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:24 smithi045 ceph-mon[32520]: pgmap v228: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:26 smithi071 ceph-mon[32578]: pgmap v229: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:26.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:26 smithi037 ceph-mon[27906]: pgmap v229: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:26 smithi045 ceph-mon[32520]: pgmap v229: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:28 smithi071 ceph-mon[32578]: pgmap v230: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:28 smithi037 ceph-mon[27906]: pgmap v230: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:28 smithi045 ceph-mon[32520]: pgmap v230: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:30 smithi071 ceph-mon[32578]: pgmap v231: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:30 smithi037 ceph-mon[27906]: pgmap v231: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:30 smithi045 ceph-mon[32520]: pgmap v231: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:32 smithi071 ceph-mon[32578]: pgmap v232: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:32 smithi037 ceph-mon[27906]: pgmap v232: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:32 smithi045 ceph-mon[32520]: pgmap v232: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:34 smithi071 ceph-mon[32578]: pgmap v233: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:34 smithi037 ceph-mon[27906]: pgmap v233: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:34 smithi045 ceph-mon[32520]: pgmap v233: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:36 smithi071 ceph-mon[32578]: pgmap v234: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:36 smithi037 ceph-mon[27906]: pgmap v234: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:36 smithi045 ceph-mon[32520]: pgmap v234: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:38 smithi071 ceph-mon[32578]: pgmap v235: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:38 smithi037 ceph-mon[27906]: pgmap v235: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:38 smithi045 ceph-mon[32520]: pgmap v235: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:40 smithi071 ceph-mon[32578]: pgmap v236: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:40.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:40 smithi037 ceph-mon[27906]: pgmap v236: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:40.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:40 smithi045 ceph-mon[32520]: pgmap v236: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:42 smithi071 ceph-mon[32578]: pgmap v237: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:42 smithi037 ceph-mon[27906]: pgmap v237: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:42 smithi045 ceph-mon[32520]: pgmap v237: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:44 smithi071 ceph-mon[32578]: pgmap v238: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:44.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:44 smithi037 ceph-mon[27906]: pgmap v238: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:44 smithi045 ceph-mon[32520]: pgmap v238: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:46 smithi071 ceph-mon[32578]: pgmap v239: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:46 smithi037 ceph-mon[27906]: pgmap v239: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:46 smithi045 ceph-mon[32520]: pgmap v239: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:48 smithi071 ceph-mon[32578]: pgmap v240: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:48.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:48 smithi037 ceph-mon[27906]: pgmap v240: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:48.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:48 smithi045 ceph-mon[32520]: pgmap v240: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:50 smithi071 ceph-mon[32578]: pgmap v241: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:50.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:50 smithi037 ceph-mon[27906]: pgmap v241: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:50 smithi045 ceph-mon[32520]: pgmap v241: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:52 smithi071 ceph-mon[32578]: pgmap v242: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:52 smithi037 ceph-mon[27906]: pgmap v242: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:52 smithi045 ceph-mon[32520]: pgmap v242: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:54 smithi071 ceph-mon[32578]: pgmap v243: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:54.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:54 smithi037 ceph-mon[27906]: pgmap v243: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:54 smithi045 ceph-mon[32520]: pgmap v243: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:05:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:56 smithi071 ceph-mon[32578]: pgmap v244: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:56 smithi037 ceph-mon[27906]: pgmap v244: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:56 smithi045 ceph-mon[32520]: pgmap v244: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:05:58 smithi071 ceph-mon[32578]: pgmap v245: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:58.462 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:05:58 smithi037 ceph-mon[27906]: pgmap v245: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:05:58.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:05:58 smithi045 ceph-mon[32520]: pgmap v245: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:00 smithi071 ceph-mon[32578]: pgmap v246: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:00 smithi037 ceph-mon[27906]: pgmap v246: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:00 smithi045 ceph-mon[32520]: pgmap v246: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:02 smithi071 ceph-mon[32578]: pgmap v247: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:02 smithi037 ceph-mon[27906]: pgmap v247: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:02 smithi045 ceph-mon[32520]: pgmap v247: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:04 smithi037 ceph-mon[27906]: pgmap v248: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:04 smithi045 ceph-mon[32520]: pgmap v248: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:04.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:04 smithi071 ceph-mon[32578]: pgmap v248: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:06.454 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:06 smithi071 ceph-mon[32578]: pgmap v249: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:06:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:06 smithi037 ceph-mon[27906]: pgmap v249: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:06:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:06 smithi045 ceph-mon[32520]: pgmap v249: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:06:08.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:08 smithi037 ceph-mon[27906]: pgmap v250: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:06:08.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:08 smithi045 ceph-mon[32520]: pgmap v250: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:06:08.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:08 smithi071 ceph-mon[32578]: pgmap v250: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 18 op/s 2024-04-26T02:06:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:10 smithi037 ceph-mon[27906]: pgmap v251: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:10.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:10 smithi045 ceph-mon[32520]: pgmap v251: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:10.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:10 smithi071 ceph-mon[32578]: pgmap v251: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:12.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:12 smithi037 ceph-mon[27906]: pgmap v252: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:12.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:12 smithi037 ceph-mon[27906]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T02:06:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:12 smithi045 ceph-mon[32520]: pgmap v252: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:12 smithi045 ceph-mon[32520]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T02:06:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:12 smithi071 ceph-mon[32578]: pgmap v252: 97 pgs: 97 active+clean; 15 GiB data, 45 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-04-26T02:06:12.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:12 smithi071 ceph-mon[32578]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T02:06:14.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:14 smithi037 ceph-mon[27906]: pgmap v254: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:06:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:14 smithi045 ceph-mon[32520]: pgmap v254: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:06:14.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:14 smithi071 ceph-mon[32578]: pgmap v254: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:06:16.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:15 smithi071 ceph-mon[32578]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T02:06:16.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:15 smithi071 ceph-mon[32578]: pgmap v256: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:06:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:15 smithi037 ceph-mon[27906]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T02:06:16.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:15 smithi037 ceph-mon[27906]: pgmap v256: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:06:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:15 smithi045 ceph-mon[32520]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T02:06:16.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:15 smithi045 ceph-mon[32520]: pgmap v256: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:06:18.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:17 smithi037 ceph-mon[27906]: pgmap v257: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:06:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:17 smithi045 ceph-mon[32520]: pgmap v257: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:06:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:17 smithi071 ceph-mon[32578]: pgmap v257: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:06:19.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:18 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:06:19.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:18 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:19.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:18 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:19.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:18 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:06:19.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:18 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:19.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:18 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:19.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:18 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:06:19.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:18 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:19.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:18 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:20.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:19 smithi037 ceph-mon[27906]: pgmap v258: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 383 B/s wr, 3 op/s 2024-04-26T02:06:20.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:19 smithi045 ceph-mon[32520]: pgmap v258: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 383 B/s wr, 3 op/s 2024-04-26T02:06:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:19 smithi071 ceph-mon[32578]: pgmap v258: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 383 B/s wr, 3 op/s 2024-04-26T02:06:21.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:20 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:06:21.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:20 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:06:21.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:20 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:21.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:20 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:06:21.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:20 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:06:21.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:20 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:20 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:06:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:20 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:06:21.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:20 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:06:22.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:22 smithi071 ceph-mon[32578]: pgmap v259: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 316 B/s wr, 2 op/s 2024-04-26T02:06:22.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:22 smithi037 ceph-mon[27906]: pgmap v259: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 316 B/s wr, 2 op/s 2024-04-26T02:06:22.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:22 smithi045 ceph-mon[32520]: pgmap v259: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 316 B/s wr, 2 op/s 2024-04-26T02:06:24.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:24 smithi071 ceph-mon[32578]: pgmap v260: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:06:24.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:24 smithi037 ceph-mon[27906]: pgmap v260: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:06:24.754 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:24 smithi045 ceph-mon[32520]: pgmap v260: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:06:26.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:25 smithi071 ceph-mon[32578]: pgmap v261: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 203 B/s wr, 2 op/s 2024-04-26T02:06:26.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:25 smithi037 ceph-mon[27906]: pgmap v261: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 203 B/s wr, 2 op/s 2024-04-26T02:06:26.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:25 smithi045 ceph-mon[32520]: pgmap v261: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 203 B/s wr, 2 op/s 2024-04-26T02:06:28.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:27 smithi037 ceph-mon[27906]: pgmap v262: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:06:28.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:27 smithi045 ceph-mon[32520]: pgmap v262: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:06:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:27 smithi071 ceph-mon[32578]: pgmap v262: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-04-26T02:06:30.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:29 smithi037 ceph-mon[27906]: pgmap v263: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T02:06:30.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:29 smithi045 ceph-mon[32520]: pgmap v263: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T02:06:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:29 smithi071 ceph-mon[32578]: pgmap v263: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 170 B/s wr, 2 op/s 2024-04-26T02:06:32.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:31 smithi045 ceph-mon[32520]: pgmap v264: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:31 smithi071 ceph-mon[32578]: pgmap v264: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:31 smithi037 ceph-mon[27906]: pgmap v264: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:34.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:33 smithi045 ceph-mon[32520]: pgmap v265: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:33 smithi071 ceph-mon[32578]: pgmap v265: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:33 smithi037 ceph-mon[27906]: pgmap v265: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:36.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:35 smithi045 ceph-mon[32520]: pgmap v266: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:36.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:35 smithi071 ceph-mon[32578]: pgmap v266: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:35 smithi037 ceph-mon[27906]: pgmap v266: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:38.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:37 smithi045 ceph-mon[32520]: pgmap v267: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:37 smithi071 ceph-mon[32578]: pgmap v267: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:37 smithi037 ceph-mon[27906]: pgmap v267: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:40.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:39 smithi045 ceph-mon[32520]: pgmap v268: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:39 smithi071 ceph-mon[32578]: pgmap v268: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:40.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:39 smithi037 ceph-mon[27906]: pgmap v268: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:42.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:41 smithi045 ceph-mon[32520]: pgmap v269: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:41 smithi071 ceph-mon[32578]: pgmap v269: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:41 smithi037 ceph-mon[27906]: pgmap v269: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:44.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:43 smithi045 ceph-mon[32520]: pgmap v270: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:44.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:43 smithi071 ceph-mon[32578]: pgmap v270: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:43 smithi037 ceph-mon[27906]: pgmap v270: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:46.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:45 smithi045 ceph-mon[32520]: pgmap v271: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:45 smithi071 ceph-mon[32578]: pgmap v271: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:45 smithi037 ceph-mon[27906]: pgmap v271: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:48.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:48 smithi045 ceph-mon[32520]: pgmap v272: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:48.261 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:48 smithi037 ceph-mon[27906]: pgmap v272: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:48 smithi071 ceph-mon[32578]: pgmap v272: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:50 smithi071 ceph-mon[32578]: pgmap v273: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:50.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:50 smithi037 ceph-mon[27906]: pgmap v273: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:50 smithi045 ceph-mon[32520]: pgmap v273: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:52 smithi071 ceph-mon[32578]: pgmap v274: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:52 smithi037 ceph-mon[27906]: pgmap v274: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:52 smithi045 ceph-mon[32520]: pgmap v274: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:54 smithi071 ceph-mon[32578]: pgmap v275: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:54.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:54 smithi037 ceph-mon[27906]: pgmap v275: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:54 smithi045 ceph-mon[32520]: pgmap v275: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:06:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:56 smithi071 ceph-mon[32578]: pgmap v276: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:56 smithi037 ceph-mon[27906]: pgmap v276: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:56.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:56 smithi045 ceph-mon[32520]: pgmap v276: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:06:58 smithi071 ceph-mon[32578]: pgmap v277: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:06:58 smithi037 ceph-mon[27906]: pgmap v277: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:06:58.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:06:58 smithi045 ceph-mon[32520]: pgmap v277: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:00.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:00 smithi071 ceph-mon[32578]: pgmap v278: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:00 smithi037 ceph-mon[27906]: pgmap v278: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:00 smithi045 ceph-mon[32520]: pgmap v278: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:02.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:02 smithi071 ceph-mon[32578]: pgmap v279: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T02:07:02.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:02 smithi037 ceph-mon[27906]: pgmap v279: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T02:07:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:02 smithi045 ceph-mon[32520]: pgmap v279: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-04-26T02:07:04.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:04 smithi071 ceph-mon[32578]: pgmap v280: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T02:07:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:04 smithi037 ceph-mon[27906]: pgmap v280: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T02:07:04.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:04 smithi045 ceph-mon[32520]: pgmap v280: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T02:07:06.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:06 smithi071 ceph-mon[32578]: pgmap v281: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:06 smithi037 ceph-mon[27906]: pgmap v281: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:06 smithi045 ceph-mon[32520]: pgmap v281: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:08.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:08 smithi071 ceph-mon[32578]: pgmap v282: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:08.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:08 smithi037 ceph-mon[27906]: pgmap v282: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:08.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:08 smithi045 ceph-mon[32520]: pgmap v282: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:10.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:10 smithi071 ceph-mon[32578]: pgmap v283: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T02:07:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:10 smithi037 ceph-mon[27906]: pgmap v283: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T02:07:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:10 smithi045 ceph-mon[32520]: pgmap v283: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 44 KiB/s wr, 19 op/s 2024-04-26T02:07:12.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:12 smithi071 ceph-mon[32578]: pgmap v284: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:12.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:12 smithi071 ceph-mon[32578]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T02:07:12.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:12 smithi037 ceph-mon[27906]: pgmap v284: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:12.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:12 smithi037 ceph-mon[27906]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T02:07:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:12 smithi045 ceph-mon[32520]: pgmap v284: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T02:07:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:12 smithi045 ceph-mon[32520]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T02:07:14.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:14 smithi071 ceph-mon[32578]: pgmap v286: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:07:14.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:14 smithi037 ceph-mon[27906]: pgmap v286: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:07:14.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:14 smithi045 ceph-mon[32520]: pgmap v286: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:07:16.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:15 smithi071 ceph-mon[32578]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T02:07:16.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:15 smithi071 ceph-mon[32578]: pgmap v288: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:07:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:15 smithi037 ceph-mon[27906]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T02:07:16.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:15 smithi037 ceph-mon[27906]: pgmap v288: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:07:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:15 smithi045 ceph-mon[32520]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T02:07:16.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:15 smithi045 ceph-mon[32520]: pgmap v288: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:07:18.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:17 smithi045 ceph-mon[32520]: pgmap v289: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:07:18.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:17 smithi071 ceph-mon[32578]: pgmap v289: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:07:18.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:17 smithi037 ceph-mon[27906]: pgmap v289: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:07:20.257 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:19 smithi045 ceph-mon[32520]: pgmap v290: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:07:20.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:19 smithi071 ceph-mon[32578]: pgmap v290: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:07:20.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:19 smithi037 ceph-mon[27906]: pgmap v290: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:07:21.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:20 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:07:21.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:20 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:07:21.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:20 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:07:22.254 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:21 smithi045 ceph-mon[32520]: pgmap v291: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 208 B/s wr, 2 op/s 2024-04-26T02:07:22.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:21 smithi071 ceph-mon[32578]: pgmap v291: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 208 B/s wr, 2 op/s 2024-04-26T02:07:22.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:21 smithi037 ceph-mon[27906]: pgmap v291: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 208 B/s wr, 2 op/s 2024-04-26T02:07:23.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:22 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:07:23.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:22 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:07:23.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:22 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:07:23.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:22 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:07:23.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:22 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:07:23.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:22 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:07:23.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:22 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:07:23.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:22 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:07:23.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:22 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:07:24.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:24 smithi071 ceph-mon[32578]: pgmap v292: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:07:24.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:24 smithi037 ceph-mon[27906]: pgmap v292: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:07:24.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:24 smithi045 ceph-mon[32520]: pgmap v292: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:07:26.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:26 smithi071 ceph-mon[32578]: pgmap v293: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 101 B/s wr, 2 op/s 2024-04-26T02:07:26.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:26 smithi037 ceph-mon[27906]: pgmap v293: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 101 B/s wr, 2 op/s 2024-04-26T02:07:26.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:26 smithi045 ceph-mon[32520]: pgmap v293: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 101 B/s wr, 2 op/s 2024-04-26T02:07:28.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:28 smithi071 ceph-mon[32578]: pgmap v294: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:07:28.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:28 smithi037 ceph-mon[27906]: pgmap v294: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:07:28.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:28 smithi045 ceph-mon[32520]: pgmap v294: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-04-26T02:07:30.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:30 smithi071 ceph-mon[32578]: pgmap v295: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:07:30.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:30 smithi037 ceph-mon[27906]: pgmap v295: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:07:30.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:30 smithi045 ceph-mon[32520]: pgmap v295: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:07:32.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:32 smithi071 ceph-mon[32578]: pgmap v296: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:32 smithi037 ceph-mon[27906]: pgmap v296: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:32.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:32 smithi045 ceph-mon[32520]: pgmap v296: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:34.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:34 smithi071 ceph-mon[32578]: pgmap v297: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:34.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:34 smithi037 ceph-mon[27906]: pgmap v297: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:34.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:34 smithi045 ceph-mon[32520]: pgmap v297: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:36.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:36 smithi071 ceph-mon[32578]: pgmap v298: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:36.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:36 smithi037 ceph-mon[27906]: pgmap v298: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:36 smithi045 ceph-mon[32520]: pgmap v298: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:38 smithi071 ceph-mon[32578]: pgmap v299: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:38 smithi037 ceph-mon[27906]: pgmap v299: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:38.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:38 smithi045 ceph-mon[32520]: pgmap v299: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:40.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:40 smithi071 ceph-mon[32578]: pgmap v300: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:40.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:40 smithi037 ceph-mon[27906]: pgmap v300: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:40.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:40 smithi045 ceph-mon[32520]: pgmap v300: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:42.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:42 smithi071 ceph-mon[32578]: pgmap v301: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:42.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:42 smithi037 ceph-mon[27906]: pgmap v301: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:42.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:42 smithi045 ceph-mon[32520]: pgmap v301: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:44.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:44 smithi071 ceph-mon[32578]: pgmap v302: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:44.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:44 smithi037 ceph-mon[27906]: pgmap v302: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:44.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:44 smithi045 ceph-mon[32520]: pgmap v302: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:46.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:46 smithi071 ceph-mon[32578]: pgmap v303: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:46.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:46 smithi037 ceph-mon[27906]: pgmap v303: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:46.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:46 smithi045 ceph-mon[32520]: pgmap v303: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:48.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:48 smithi071 ceph-mon[32578]: pgmap v304: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:48.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:48 smithi037 ceph-mon[27906]: pgmap v304: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:48.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:48 smithi045 ceph-mon[32520]: pgmap v304: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:50.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:50 smithi071 ceph-mon[32578]: pgmap v305: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:50.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:50 smithi037 ceph-mon[27906]: pgmap v305: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:50.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:50 smithi045 ceph-mon[32520]: pgmap v305: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:52.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:52 smithi071 ceph-mon[32578]: pgmap v306: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:52.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:52 smithi037 ceph-mon[27906]: pgmap v306: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:52.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:52 smithi045 ceph-mon[32520]: pgmap v306: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:54.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:54 smithi071 ceph-mon[32578]: pgmap v307: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:54.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:54 smithi037 ceph-mon[27906]: pgmap v307: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:54.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:54 smithi045 ceph-mon[32520]: pgmap v307: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:07:56.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:56 smithi071 ceph-mon[32578]: pgmap v308: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:56.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:56 smithi037 ceph-mon[27906]: pgmap v308: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:56 smithi045 ceph-mon[32520]: pgmap v308: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:58.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:07:58 smithi071 ceph-mon[32578]: pgmap v309: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:58.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:07:58 smithi037 ceph-mon[27906]: pgmap v309: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:58.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:07:58 smithi045 ceph-mon[32520]: pgmap v309: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:07:59.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T02:07:59.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 -- 172.21.15.37:0/3293570539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89410e6f0 msgr2=0x7fb894110ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 --2- 172.21.15.37:0/3293570539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89410e6f0 0x7fb894110ae0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb88c009860 tx=0x7fb88c02a340 comp rx=0 tx=0).stop 2024-04-26T02:07:59.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 -- 172.21.15.37:0/3293570539 shutdown_connections 2024-04-26T02:07:59.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 --2- 172.21.15.37:0/3293570539 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb89410e6f0 0x7fb894110ae0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 --2- 172.21.15.37:0/3293570539 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8941014f0 0x7fb89410e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 --2- 172.21.15.37:0/3293570539 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb894100b40 0x7fb894100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 -- 172.21.15.37:0/3293570539 >> 172.21.15.37:0/3293570539 conn(0x7fb8940fc730 msgr2=0x7fb8940feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:07:59.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 -- 172.21.15.37:0/3293570539 shutdown_connections 2024-04-26T02:07:59.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.420+0000 7fb893e00640 1 -- 172.21.15.37:0/3293570539 wait complete. 2024-04-26T02:07:59.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.421+0000 7fb893e00640 1 Processor -- start 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.421+0000 7fb893e00640 1 -- start start 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 0x7fb8941a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8941014f0 0x7fb8941a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893e00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89410e6f0 0x7fb89419a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893e00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8941141d0 con 0x7fb894100b40 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893e00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fb894114050 con 0x7fb8941014f0 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893e00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fb894114350 con 0x7fb89410e6f0 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb892a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 0x7fb8941a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb892a00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 0x7fb8941a0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60424/0 (socket says 172.21.15.37:60424) 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb892a00640 1 -- 172.21.15.37:0/1534784527 learned_addr learned my addr 172.21.15.37:0/1534784527 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb893400640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89410e6f0 0x7fb89419a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.422+0000 7fb892000640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8941014f0 0x7fb8941a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892a00640 1 -- 172.21.15.37:0/1534784527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89410e6f0 msgr2=0x7fb89419a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892a00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89410e6f0 0x7fb89419a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892a00640 1 -- 172.21.15.37:0/1534784527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8941014f0 msgr2=0x7fb8941a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892a00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8941014f0 0x7fb8941a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892a00640 1 -- 172.21.15.37:0/1534784527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89419ad30 con 0x7fb894100b40 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb893400640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89410e6f0 0x7fb89419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892000640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8941014f0 0x7fb8941a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.423+0000 7fb892a00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 0x7fb8941a0400 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb884009d50 tx=0x7fb88400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.424+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884003b70 con 0x7fb894100b40 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.424+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb89419b020 con 0x7fb894100b40 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.424+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb884003d10 con 0x7fb894100b40 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.424+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb884004d40 con 0x7fb894100b40 2024-04-26T02:07:59.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.424+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941a72c0 con 0x7fb894100b40 2024-04-26T02:07:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.425+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8940687f0 con 0x7fb894100b40 2024-04-26T02:07:59.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.427+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb884004000 con 0x7fb894100b40 2024-04-26T02:07:59.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.429+0000 7fb88b400640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8780a3d00 0x7fb8780a61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.429+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7fb8840d0e30 con 0x7fb894100b40 2024-04-26T02:07:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.429+0000 7fb892000640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8780a3d00 0x7fb8780a61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.430+0000 7fb892000640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8780a3d00 0x7fb8780a61c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0096c0 tx=0x7fb87c009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:07:59.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.435+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb88409d470 con 0x7fb894100b40 2024-04-26T02:07:59.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.670+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb894062390 con 0x7fb8780a3d00 2024-04-26T02:07:59.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.685+0000 7fb88b400640 1 -- 172.21.15.37:0/1534784527 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb894062390 con 0x7fb8780a3d00 2024-04-26T02:07:59.684 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T01:57:57", "first": "2024-04-26T01:58:00", "last": "2024-04-26T02:07:00", "last_pruned": "2024-04-26T02:07:01", "created_count": 10, "pruned_count": 3, "active": true} 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8780a3d00 msgr2=0x7fb8780a61c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8780a3d00 0x7fb8780a61c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb87c0096c0 tx=0x7fb87c009260 comp rx=0 tx=0).stop 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 msgr2=0x7fb8941a0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 0x7fb8941a0400 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb884009d50 tx=0x7fb88400c6a0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 shutdown_connections 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fb8780a3d00 0x7fb8780a61c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fb89410e6f0 0x7fb89419a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fb8941014f0 0x7fb8941a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 --2- 172.21.15.37:0/1534784527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb894100b40 0x7fb8941a0400 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.688+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 >> 172.21.15.37:0/1534784527 conn(0x7fb8940fc730 msgr2=0x7fb8940feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:07:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.689+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 shutdown_connections 2024-04-26T02:07:59.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.689+0000 7fb893e00640 1 -- 172.21.15.37:0/1534784527 wait complete. 2024-04-26T02:07:59.708 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T02:07:59.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 -- 172.21.15.37:0/2732894505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076480 msgr2=0x7f7b14076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/2732894505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076480 0x7f7b14076860 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009930 tx=0x7f7b0802a440 comp rx=0 tx=0).stop 2024-04-26T02:07:59.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 -- 172.21.15.37:0/2732894505 shutdown_connections 2024-04-26T02:07:59.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/2732894505 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b14112c20 0x7f7b14115010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/2732894505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14076da0 0x7f7b141126e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/2732894505 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076480 0x7f7b14076860 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 -- 172.21.15.37:0/2732894505 >> 172.21.15.37:0/2732894505 conn(0x7f7b140ffa50 msgr2=0x7f7b14101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 -- 172.21.15.37:0/2732894505 shutdown_connections 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.923+0000 7f7b1ac00640 1 -- 172.21.15.37:0/2732894505 wait complete. 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.925+0000 7f7b1ac00640 1 Processor -- start 2024-04-26T02:07:59.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.925+0000 7f7b1ac00640 1 -- start start 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1ac00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b14076480 0x7f7b141a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1ac00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076da0 0x7f7b141a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1ac00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 0x7f7b1419ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1ac00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b14118690 con 0x7f7b14112c20 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1ac00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f7b14118510 con 0x7f7b14076da0 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1ac00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f7b14118810 con 0x7f7b14076480 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 0x7f7b1419ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b18e00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076da0 0x7f7b141a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1a200640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 0x7f7b1419ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60432/0 (socket says 172.21.15.37:60432) 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b1a200640 1 -- 172.21.15.37:0/3666416172 learned_addr learned my addr 172.21.15.37:0/3666416172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:07:59.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.926+0000 7f7b19800640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b14076480 0x7f7b141a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1a200640 1 -- 172.21.15.37:0/3666416172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b14076480 msgr2=0x7f7b141a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1a200640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b14076480 0x7f7b141a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1a200640 1 -- 172.21.15.37:0/3666416172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076da0 msgr2=0x7f7b141a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1a200640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076da0 0x7f7b141a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1a200640 1 -- 172.21.15.37:0/3666416172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1419f1b0 con 0x7f7b14112c20 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b18e00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076da0 0x7f7b141a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1a200640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 0x7f7b1419ea30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00eb40 tx=0x7f7b0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c004b20 con 0x7f7b14112c20 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b0c004cc0 con 0x7f7b14112c20 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1419f4a0 con 0x7f7b14112c20 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.927+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0c010720 con 0x7f7b14112c20 2024-04-26T02:07:59.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.928+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b141ab740 con 0x7f7b14112c20 2024-04-26T02:07:59.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.929+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b140687f0 con 0x7f7b14112c20 2024-04-26T02:07:59.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.931+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7b0c005560 con 0x7f7b14112c20 2024-04-26T02:07:59.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.932+0000 7f7b12000640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7afc0a3d80 0x7f7afc0a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:07:59.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.932+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f7b0c0d0ea0 con 0x7f7b14112c20 2024-04-26T02:07:59.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.933+0000 7f7b19800640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7afc0a3d80 0x7f7afc0a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:07:59.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.933+0000 7f7b19800640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7afc0a3d80 0x7f7afc0a6240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b08031440 tx=0x7f7b08005a20 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:07:59.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:07:59.937+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b0c09d4e0 con 0x7f7b14112c20 2024-04-26T02:08:00.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.179+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 --> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f7b14062390 con 0x7f7afc0a3d80 2024-04-26T02:08:00.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.287+0000 7f7b12000640 1 -- 172.21.15.37:0/3666416172 <== mgr.14856 v2:172.21.15.37:6800/3717150907 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f7b14062390 con 0x7f7afc0a3d80 2024-04-26T02:08:00.286 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-04-26T02:08:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7afc0a3d80 msgr2=0x7f7afc0a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7afc0a3d80 0x7f7afc0a6240 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7b08031440 tx=0x7f7b08005a20 comp rx=0 tx=0).stop 2024-04-26T02:08:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 msgr2=0x7f7b1419ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 0x7f7b1419ea30 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7b0c00eb40 tx=0x7f7b0c00c2d0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 shutdown_connections 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f7afc0a3d80 0x7f7afc0a6240 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b14112c20 0x7f7b1419ea30 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f7b14076da0 0x7f7b141a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 --2- 172.21.15.37:0/3666416172 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f7b14076480 0x7f7b141a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 >> 172.21.15.37:0/3666416172 conn(0x7f7b140ffa50 msgr2=0x7f7b14101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 shutdown_connections 2024-04-26T02:08:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.290+0000 7f7b1ac00640 1 -- 172.21.15.37:0/3666416172 wait complete. 2024-04-26T02:08:00.308 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T02:08:00.320 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T02:08:00.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:00 smithi037 ceph-mon[27906]: pgmap v310: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:00.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:00 smithi045 ceph-mon[32520]: pgmap v310: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.511+0000 7f3c49400640 1 -- 172.21.15.37:0/3608940494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c441043b0 msgr2=0x7f3c44104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.511+0000 7f3c49400640 1 --2- 172.21.15.37:0/3608940494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c441043b0 0x7f3c44104790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3c34009be0 tx=0x7f3c3402a6c0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 -- 172.21.15.37:0/3608940494 shutdown_connections 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 --2- 172.21.15.37:0/3608940494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c44111050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 --2- 172.21.15.37:0/3608940494 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c44104d60 0x7f3c44108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 --2- 172.21.15.37:0/3608940494 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c441043b0 0x7f3c44104790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 -- 172.21.15.37:0/3608940494 >> 172.21.15.37:0/3608940494 conn(0x7f3c440ffc10 msgr2=0x7f3c44102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 -- 172.21.15.37:0/3608940494 shutdown_connections 2024-04-26T02:08:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.512+0000 7f3c49400640 1 -- 172.21.15.37:0/3608940494 wait complete. 2024-04-26T02:08:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 Processor -- start 2024-04-26T02:08:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 -- start start 2024-04-26T02:08:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c441043b0 0x7f3c441a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c441a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c4419a760 0x7f3c4419abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c441143c0 con 0x7f3c44109480 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44114240 con 0x7f3c441043b0 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c49400640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f3c44114540 con 0x7f3c4419a760 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c39e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c441a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.513+0000 7f3c39e00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c441a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47020/0 (socket says 172.21.15.37:47020) 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 -- 172.21.15.37:0/3713323696 learned_addr learned my addr 172.21.15.37:0/3713323696 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c43200640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c4419a760 0x7f3c4419abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c42800640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c441043b0 0x7f3c441a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 -- 172.21.15.37:0/3713323696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c4419a760 msgr2=0x7f3c4419abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c4419a760 0x7f3c4419abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 -- 172.21.15.37:0/3713323696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c441043b0 msgr2=0x7f3c441a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c441043b0 0x7f3c441a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 -- 172.21.15.37:0/3713323696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c4419b360 con 0x7f3c44109480 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c39e00640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c441a0be0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e9a0 tx=0x7f3c2c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.514+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00cd80 con 0x7f3c44109480 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.515+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2c0035f0 con 0x7f3c44109480 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.515+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c441a7440 con 0x7f3c44109480 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.515+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c010690 con 0x7f3c44109480 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.515+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c441a78a0 con 0x7f3c44109480 2024-04-26T02:08:00.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.516+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c44105880 con 0x7f3c44109480 2024-04-26T02:08:00.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.518+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c2c004000 con 0x7f3c44109480 2024-04-26T02:08:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.519+0000 7f3c3be00640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f3c280a3da0 0x7f3c280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.519+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f3c2c0d0af0 con 0x7f3c44109480 2024-04-26T02:08:00.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.519+0000 7f3c42800640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f3c280a3da0 0x7f3c280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.520+0000 7f3c42800640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f3c280a3da0 0x7f3c280a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c34000f80 tx=0x7f3c34009310 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:00.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.524+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c2c0042f0 con 0x7f3c44109480 2024-04-26T02:08:00.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:00 smithi071 ceph-mon[32578]: pgmap v310: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:00.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.732+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c44069250 con 0x7f3c44109480 2024-04-26T02:08:00.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.736+0000 7f3c3be00640 1 -- 172.21.15.37:0/3713323696 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+22961 (secure 0 0 0) 0x7f3c44069250 con 0x7f3c44109480 2024-04-26T02:08:00.735 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:08:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":98,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T02:07:14.843833+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":97,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4056686696":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/793487795":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/29145428":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/457714458":"2024-04-27T01:57:56.610509+0000","172.21.15.37:6800/691602839":"2024-04-27T01:57:56.610509+0000","172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/691602839":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:08:00.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f3c280a3da0 msgr2=0x7f3c280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f3c280a3da0 0x7f3c280a6260 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3c34000f80 tx=0x7f3c34009310 comp rx=0 tx=0).stop 2024-04-26T02:08:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 msgr2=0x7f3c441a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c441a0be0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e9a0 tx=0x7f3c2c00ee60 comp rx=0 tx=0).stop 2024-04-26T02:08:00.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 shutdown_connections 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f3c280a3da0 0x7f3c280a6260 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f3c4419a760 0x7f3c4419abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c44109480 0x7f3c441a0be0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 --2- 172.21.15.37:0/3713323696 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f3c441043b0 0x7f3c441a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.739+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 >> 172.21.15.37:0/3713323696 conn(0x7f3c440ffc10 msgr2=0x7f3c44100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.740+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 shutdown_connections 2024-04-26T02:08:00.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.740+0000 7f3c49400640 1 -- 172.21.15.37:0/3713323696 wait complete. 2024-04-26T02:08:00.759 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T01:48:32.516892+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T01:54:02.892469+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T01:54:06.974700+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 97, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T02:08:00.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T02:08:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 -- 172.21.15.37:0/1200377139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076da0 msgr2=0x7f882c112690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 --2- 172.21.15.37:0/1200377139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076da0 0x7f882c112690 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8818009900 tx=0x7f881802a410 comp rx=0 tx=0).stop 2024-04-26T02:08:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 -- 172.21.15.37:0/1200377139 shutdown_connections 2024-04-26T02:08:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 --2- 172.21.15.37:0/1200377139 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f882c112bd0 0x7f882c114fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 --2- 172.21.15.37:0/1200377139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076da0 0x7f882c112690 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 --2- 172.21.15.37:0/1200377139 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f882c076480 0x7f882c076860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 -- 172.21.15.37:0/1200377139 >> 172.21.15.37:0/1200377139 conn(0x7f882c0ffa20 msgr2=0x7f882c101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 -- 172.21.15.37:0/1200377139 shutdown_connections 2024-04-26T02:08:00.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.977+0000 7f8831600640 1 -- 172.21.15.37:0/1200377139 wait complete. 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.978+0000 7f8831600640 1 Processor -- start 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 -- start start 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 0x7f882c1a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f882c076da0 0x7f882c1a4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f882c112bd0 0x7f882c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f882c118620 con 0x7f882c076480 2024-04-26T02:08:00.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1184a0 con 0x7f882c076da0 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8831600640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f882c1187a0 con 0x7f882c112bd0 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f882be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 0x7f882c1a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f882be00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 0x7f882c1a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47032/0 (socket says 172.21.15.37:47032) 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f882be00640 1 -- 172.21.15.37:0/2790455482 learned_addr learned my addr 172.21.15.37:0/2790455482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f8830c00640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f882c112bd0 0x7f882c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.979+0000 7f882b400640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f882c076da0 0x7f882c1a4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f882be00640 1 -- 172.21.15.37:0/2790455482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f882c112bd0 msgr2=0x7f882c19ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f882be00640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f882c112bd0 0x7f882c19ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f882be00640 1 -- 172.21.15.37:0/2790455482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f882c076da0 msgr2=0x7f882c1a4df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f882be00640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f882c076da0 0x7f882c1a4df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f882be00640 1 -- 172.21.15.37:0/2790455482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882c19f1e0 con 0x7f882c076480 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f882be00640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 0x7f882c1a48b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e9f0 tx=0x7f881c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c00cd80 con 0x7f882c076480 2024-04-26T02:08:00.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f881c0108e0 con 0x7f882c076480 2024-04-26T02:08:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f881c018980 con 0x7f882c076480 2024-04-26T02:08:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.980+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882c19f4d0 con 0x7f882c076480 2024-04-26T02:08:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.981+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f882c1ab770 con 0x7f882c076480 2024-04-26T02:08:00.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.982+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f882c04fce0 con 0x7f882c076480 2024-04-26T02:08:00.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.984+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f881c010430 con 0x7f882c076480 2024-04-26T02:08:00.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.985+0000 7f8828c00640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f88100a3d20 0x7f88100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:00.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.985+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f881c0d0b40 con 0x7f882c076480 2024-04-26T02:08:00.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.985+0000 7f882b400640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f88100a3d20 0x7f88100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:00.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.986+0000 7f882b400640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f88100a3d20 0x7f88100a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f8818005e80 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:00.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:00.990+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f881c09cb30 con 0x7f882c076480 2024-04-26T02:08:01.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.205+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f882c10dd60 con 0x7f882c076480 2024-04-26T02:08:01.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.206+0000 7f8828c00640 1 -- 172.21.15.37:0/2790455482 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) ==== 93+0+10 (secure 0 0 0) 0x7f881c0a0e30 con 0x7f882c076480 2024-04-26T02:08:01.206 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f88100a3d20 msgr2=0x7f88100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f88100a3d20 0x7f88100a61e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8818000c00 tx=0x7f8818005e80 comp rx=0 tx=0).stop 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 msgr2=0x7f882c1a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 0x7f882c1a48b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f881c00e9f0 tx=0x7f881c00eeb0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 shutdown_connections 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f88100a3d20 0x7f88100a61e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f882c112bd0 0x7f882c19ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f882c076da0 0x7f882c1a4df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 --2- 172.21.15.37:0/2790455482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f882c076480 0x7f882c1a48b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 >> 172.21.15.37:0/2790455482 conn(0x7f882c0ffa20 msgr2=0x7f882c1143a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:01.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 shutdown_connections 2024-04-26T02:08:01.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.209+0000 7f8831600640 1 -- 172.21.15.37:0/2790455482 wait complete. 2024-04-26T02:08:01.230 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T02:08:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 -- 172.21.15.37:0/658873725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c076480 msgr2=0x7f366c076860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 --2- 172.21.15.37:0/658873725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c076480 0x7f366c076860 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3654009930 tx=0x7f365402a440 comp rx=0 tx=0).stop 2024-04-26T02:08:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 -- 172.21.15.37:0/658873725 shutdown_connections 2024-04-26T02:08:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 --2- 172.21.15.37:0/658873725 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f366c112be0 0x7f366c114fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 --2- 172.21.15.37:0/658873725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c1126a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 --2- 172.21.15.37:0/658873725 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c076480 0x7f366c076860 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 -- 172.21.15.37:0/658873725 >> 172.21.15.37:0/658873725 conn(0x7f366c0ff9f0 msgr2=0x7f366c101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 -- 172.21.15.37:0/658873725 shutdown_connections 2024-04-26T02:08:01.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.454+0000 7f3671000640 1 -- 172.21.15.37:0/658873725 wait complete. 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.455+0000 7f3671000640 1 Processor -- start 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.455+0000 7f3671000640 1 -- start start 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f3671000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f3671000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f366c112be0 0x7f366c19f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f3671000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c1a36a0 0x7f366c1a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f3671000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f366c118600 con 0x7f366c076da0 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f3671000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f366c118480 con 0x7f366c1a36a0 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f3671000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f366c118780 con 0x7f366c112be0 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c19ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47058/0 (socket says 172.21.15.37:47058) 2024-04-26T02:08:01.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366aa00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f366c112be0 0x7f366c19f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 -- 172.21.15.37:0/1617605116 learned_addr learned my addr 172.21.15.37:0/1617605116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366be00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c1a36a0 0x7f366c1a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 -- 172.21.15.37:0/1617605116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f366c112be0 msgr2=0x7f366c19f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f366c112be0 0x7f366c19f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 -- 172.21.15.37:0/1617605116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c1a36a0 msgr2=0x7f366c1a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c1a36a0 0x7f366c1a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.456+0000 7f366b400640 1 -- 172.21.15.37:0/1617605116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c1a4190 con 0x7f366c076da0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.457+0000 7f366b400640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c19ee30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3654009230 tx=0x7f3654031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.457+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365403f030 con 0x7f366c076da0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.457+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c1a4420 con 0x7f366c076da0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.457+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1abbe0 con 0x7f366c076da0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.457+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36540043d0 con 0x7f366c076da0 2024-04-26T02:08:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.457+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3654035070 con 0x7f366c076da0 2024-04-26T02:08:01.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:01 smithi037 ceph-mon[27906]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:08:01.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:01 smithi037 ceph-mon[27906]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:08:01.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:01 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3713323696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:08:01.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.459+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3648005290 con 0x7f366c076da0 2024-04-26T02:08:01.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.461+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3654042030 con 0x7f366c076da0 2024-04-26T02:08:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.462+0000 7f3663e00640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f36500a3da0 0x7f36500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.462+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f36540f07d0 con 0x7f366c076da0 2024-04-26T02:08:01.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.463+0000 7f366aa00640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f36500a3da0 0x7f36500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.464+0000 7f366aa00640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f36500a3da0 0x7f36500a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f366c101ec0 tx=0x7f365c005f90 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:01.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.467+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3654049030 con 0x7f366c076da0 2024-04-26T02:08:01.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:01 smithi045 ceph-mon[32520]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:08:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:01 smithi045 ceph-mon[32520]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:08:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:01 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3713323696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:08:01.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:01 smithi071 ceph-mon[32578]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:08:01.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:01 smithi071 ceph-mon[32578]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T02:08:01.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:01 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3713323696' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:08:01.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.675+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3648005850 con 0x7f366c076da0 2024-04-26T02:08:01.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.676+0000 7f3663e00640 1 -- 172.21.15.37:0/1617605116 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v98) ==== 104+0+11 (secure 0 0 0) 0x7f36540bce10 con 0x7f366c076da0 2024-04-26T02:08:01.676 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f36500a3da0 msgr2=0x7f36500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f36500a3da0 0x7f36500a6260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f366c101ec0 tx=0x7f365c005f90 comp rx=0 tx=0).stop 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 msgr2=0x7f366c19ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c19ee30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3654009230 tx=0x7f3654031a50 comp rx=0 tx=0).stop 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 shutdown_connections 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f36500a3da0 0x7f36500a6260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f366c1a36a0 0x7f366c1a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f366c112be0 0x7f366c19f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 --2- 172.21.15.37:0/1617605116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f366c076da0 0x7f366c19ee30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 >> 172.21.15.37:0/1617605116 conn(0x7f366c0ff9f0 msgr2=0x7f366c113360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 shutdown_connections 2024-04-26T02:08:01.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.679+0000 7f3671000640 1 -- 172.21.15.37:0/1617605116 wait complete. 2024-04-26T02:08:01.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T02:08:01.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 -- 172.21.15.37:0/2012885379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 msgr2=0x7f23c01009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 --2- 172.21.15.37:0/2012885379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 0x7f23c01009c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23b4009930 tx=0x7f23b402a420 comp rx=0 tx=0).stop 2024-04-26T02:08:01.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 -- 172.21.15.37:0/2012885379 shutdown_connections 2024-04-26T02:08:01.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 --2- 172.21.15.37:0/2012885379 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c010d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 --2- 172.21.15.37:0/2012885379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23c0100f90 0x7f23c0104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 --2- 172.21.15.37:0/2012885379 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 0x7f23c01009c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 -- 172.21.15.37:0/2012885379 >> 172.21.15.37:0/2012885379 conn(0x7f23c0075cf0 msgr2=0x7f23c0076100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:01.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 -- 172.21.15.37:0/2012885379 shutdown_connections 2024-04-26T02:08:01.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.920+0000 7f23c8000640 1 -- 172.21.15.37:0/2012885379 wait complete. 2024-04-26T02:08:01.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.921+0000 7f23c8000640 1 Processor -- start 2024-04-26T02:08:01.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.921+0000 7f23c8000640 1 -- start start 2024-04-26T02:08:01.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c8000640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 0x7f23c01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c8000640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c01a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c8000640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23c019a7b0 0x7f23c019ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c8000640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010ff90 con 0x7f23c019a7b0 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c8000640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f23c010fe10 con 0x7f23c01005e0 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c8000640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0110110 con 0x7f23c0105500 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c01a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c4c00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c01a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42124/0 (socket says 172.21.15.37:42124) 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c4c00640 1 -- 172.21.15.37:0/3929370933 learned_addr learned my addr 172.21.15.37:0/3929370933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c6000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23c019a7b0 0x7f23c019ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.922+0000 7f23c5600640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 0x7f23c01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c4c00640 1 -- 172.21.15.37:0/3929370933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 msgr2=0x7f23c01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c4c00640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 0x7f23c01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c4c00640 1 -- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23c019a7b0 msgr2=0x7f23c019ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c4c00640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23c019a7b0 0x7f23c019ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c4c00640 1 -- 172.21.15.37:0/3929370933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c019b3b0 con 0x7f23c0105500 2024-04-26T02:08:01.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c4c00640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c01a0be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0058c0 tx=0x7f23ac009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:01.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac018030 con 0x7f23c0105500 2024-04-26T02:08:01.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01a7440 con 0x7f23c0105500 2024-04-26T02:08:01.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23ac009e00 con 0x7f23c0105500 2024-04-26T02:08:01.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23ac004a00 con 0x7f23c0105500 2024-04-26T02:08:01.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.923+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c01a78a0 con 0x7f23c0105500 2024-04-26T02:08:01.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.924+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23c0101880 con 0x7f23c0105500 2024-04-26T02:08:01.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.927+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23ac01b070 con 0x7f23c0105500 2024-04-26T02:08:01.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.928+0000 7f23be000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f23a80a3d20 0x7f23a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:01.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.928+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f23ac0d0170 con 0x7f23c0105500 2024-04-26T02:08:01.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.928+0000 7f23c5600640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f23a80a3d20 0x7f23a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:01.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.929+0000 7f23c5600640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f23a80a3d20 0x7f23a80a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f23b4004770 tx=0x7f23b4000bc0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:01.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:01.933+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23ac011070 con 0x7f23c0105500 2024-04-26T02:08:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.143+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f23c0101030 con 0x7f23c0105500 2024-04-26T02:08:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.144+0000 7f23be000640 1 -- 172.21.15.37:0/3929370933 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v98) ==== 100+0+11 (secure 0 0 0) 0x7f23ac09c7b0 con 0x7f23c0105500 2024-04-26T02:08:02.144 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-04-26T02:08:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f23a80a3d20 msgr2=0x7f23a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f23a80a3d20 0x7f23a80a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f23b4004770 tx=0x7f23b4000bc0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 msgr2=0x7f23c01a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c01a0be0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f23ac0058c0 tx=0x7f23ac009bb0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 shutdown_connections 2024-04-26T02:08:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f23a80a3d20 0x7f23a80a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23c019a7b0 0x7f23c019ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f23c0105500 0x7f23c01a0be0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 --2- 172.21.15.37:0/3929370933 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f23c01005e0 0x7f23c01a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 >> 172.21.15.37:0/3929370933 conn(0x7f23c0075cf0 msgr2=0x7f23c00fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 shutdown_connections 2024-04-26T02:08:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.148+0000 7f23c8000640 1 -- 172.21.15.37:0/3929370933 wait complete. 2024-04-26T02:08:02.167 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T02:08:02.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 -- 172.21.15.37:0/10925472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10104c40 msgr2=0x7f4d10108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:02.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/10925472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10104c40 0x7f4d10108c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8009900 tx=0x7f4cf802a3f0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 -- 172.21.15.37:0/10925472 shutdown_connections 2024-04-26T02:08:02.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/10925472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d10109360 0x7f4d10110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/10925472 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10104c40 0x7f4d10108c80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/10925472 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d10104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 -- 172.21.15.37:0/10925472 >> 172.21.15.37:0/10925472 conn(0x7f4d100ffad0 msgr2=0x7f4d10101ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:02.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 -- 172.21.15.37:0/10925472 shutdown_connections 2024-04-26T02:08:02.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.378+0000 7f4d0fe00640 1 -- 172.21.15.37:0/10925472 wait complete. 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.379+0000 7f4d0fe00640 1 Processor -- start 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.379+0000 7f4d0fe00640 1 -- start start 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0fe00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d101a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0fe00640 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10109360 0x7f4d101a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0fe00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d1019a6b0 0x7f4d1019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0fe00640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d101142d0 con 0x7f4d1019a6b0 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0fe00640 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10114150 con 0x7f4d10109360 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0fe00640 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4d10114450 con 0x7f4d10104290 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d101a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:02.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0f400640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d1019a6b0 0x7f4d1019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d101a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:42130/0 (socket says 172.21.15.37:42130) 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 -- 172.21.15.37:0/444940646 learned_addr learned my addr 172.21.15.37:0/444940646 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0e000640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10109360 0x7f4d101a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 -- 172.21.15.37:0/444940646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10109360 msgr2=0x7f4d101a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10109360 0x7f4d101a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 -- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d1019a6b0 msgr2=0x7f4d1019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d1019a6b0 0x7f4d1019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.380+0000 7f4d0ea00640 1 -- 172.21.15.37:0/444940646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1019b2b0 con 0x7f4d10104290 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d0f400640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d1019a6b0 0x7f4d1019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d0ea00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d101a05a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000ea60 tx=0x7f4d0000ee60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0000cd20 con 0x7f4d10104290 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d000035f0 con 0x7f4d10104290 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d101a7340 con 0x7f4d10104290 2024-04-26T02:08:02.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d00010690 con 0x7f4d10104290 2024-04-26T02:08:02.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.381+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d101a7740 con 0x7f4d10104290 2024-04-26T02:08:02.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.382+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d10105760 con 0x7f4d10104290 2024-04-26T02:08:02.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.384+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4d000037a0 con 0x7f4d10104290 2024-04-26T02:08:02.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.386+0000 7f4d07400640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4cf40a3d90 0x7f4cf40a6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:02.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.386+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8513+0+0 (secure 0 0 0) 0x7f4d000d0a40 con 0x7f4d10104290 2024-04-26T02:08:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.386+0000 7f4d0e000640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4cf40a3d90 0x7f4cf40a6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:02.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.387+0000 7f4d0e000640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4cf40a3d90 0x7f4cf40a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8031440 tx=0x7f4cf8005a20 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:02.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.388+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d0009d080 con 0x7f4d10104290 2024-04-26T02:08:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:02 smithi037 ceph-mon[27906]: pgmap v311: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:02 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2790455482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:02 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1617605116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:02 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3929370933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:02 smithi045 ceph-mon[32520]: pgmap v311: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:02.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:02 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2790455482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:02 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1617605116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:02 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3929370933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.650+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f4d1004fd50 con 0x7f4d10104290 2024-04-26T02:08:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:02 smithi071 ceph-mon[32578]: pgmap v311: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:02 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2790455482' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:02 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1617605116' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:02 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3929370933' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T02:08:02.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.661+0000 7f4d07400640 1 -- 172.21.15.37:0/444940646 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746062 (secure 0 0 0) 0x7f4d0009c980 con 0x7f4d10104290 2024-04-26T02:08:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:08:02.671 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:08:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-04-26T02:08:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.671 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 14856, 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3717150907 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3717150907 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/3717150907", 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-04-26T01:57:56.610943+0000", 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24947, 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-26T02:08:02.673 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.680 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-26T02:08:02.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.687 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-26T02:08:02.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T02:08:02.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-26T02:08:02.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-26T02:08:02.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:08:02.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.698 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T02:08:02.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-26T02:08:02.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:08:02.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-26T02:08:02.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.709 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:08:02.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-26T02:08:02.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.712 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-26T02:08:02.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-26T02:08:02.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:08:02.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:08:02.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-26T02:08:02.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-26T02:08:02.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.726 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:08:02.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:08:02.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:02.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.738 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-26T02:08:02.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:08:02.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.744 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-26T02:08:02.751 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.766 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:08:02.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:08:02.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.785 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:08:02.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:08:02.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:02.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:08:02.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:08:02.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:08:02.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:08:02.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.819 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-26T02:08:02.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.825 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-26T02:08:02.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.831 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.832 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.838 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-26T02:08:02.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-26T02:08:02.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:02.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.851 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T02:08:02.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-26T02:08:02.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.857 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-26T02:08:02.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.869 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.870 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.871 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.874 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.875 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.876 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.882 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.883 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.885 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.887 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.889 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.892 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.894 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-26T02:08:02.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-26T02:08:02.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-26T02:08:02.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.899 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.903 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.905 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.907 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.912 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T02:08:02.914 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.916 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.918 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.930 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.935 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-26T02:08:02.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:08:02.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.943 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.944 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.946 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:02.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.948 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:08:02.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.951 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-26T02:08:02.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:08:02.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-26T02:08:02.957 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:02.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.959 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.963 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.964 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.966 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.967 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:08:02.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.970 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.971 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:02.972 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.973 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.976 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.977 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:02.979 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.981 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.982 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.984 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.986 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.988 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.989 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-26T02:08:02.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-26T02:08:02.991 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.992 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:02.994 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:02.995 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-26T02:08:02.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:02.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.003 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.005 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.008 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.011 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.013 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.013 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.013 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.013 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.015 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.018 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.020 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.023 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.025 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.026 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.027 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.028 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-26T02:08:03.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.031 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.033 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.034 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-26T02:08:03.036 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:08:03.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.038 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.039 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.041 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.043 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.044 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.048 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.049 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.050 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.058 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.059 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.061 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.062 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.063 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.064 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.066 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.067 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-26T02:08:03.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.069 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:08:03.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:08:03.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.072 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.073 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.076 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.078 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.079 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-26T02:08:03.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.084 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24982, 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.087 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.089 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.090 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.092 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.094 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.097 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.099 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:08:03.102 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:08:03.104 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:08:03.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:08:03.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:08:03.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.108 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-26T02:08:03.114 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.118 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-26T02:08:03.119 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.121 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.122 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.123 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.124 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.125 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:08:03.126 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.131 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-26T02:08:03.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.139 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.140 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.142 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-26T02:08:03.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.144 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.146 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:08:03.149 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:08:03.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:08:03.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.153 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.155 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.157 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.158 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-26T02:08:03.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:08:03.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-26T02:08:03.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-26T02:08:03.172 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-26T02:08:03.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.174 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-26T02:08:03.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.176 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-26T02:08:03.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.178 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.179 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-26T02:08:03.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-26T02:08:03.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.182 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.184 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.185 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.187 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.188 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.190 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:08:03.191 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:08:03.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:08:03.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:08:03.194 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:08:03.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:08:03.196 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.198 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:08:03.199 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:08:03.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:08:03.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:08:03.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:08:03.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.204 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:08:03.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-26T02:08:03.207 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:08:03.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:08:03.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:08:03.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.211 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:08:03.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:08:03.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:08:03.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:08:03.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:08:03.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.217 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.224 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.225 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.226 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.229 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.233 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.234 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.236 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.237 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.238 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.239 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-26T02:08:03.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T02:08:03.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.243 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.245 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-26T02:08:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:08:03.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:03.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:03.250 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T02:08:03.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-26T02:08:03.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T02:08:03.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.255 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.258 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.261 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.264 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.267 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:08:03.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:08:03.269 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:08:03.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.271 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.272 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.273 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.275 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-26T02:08:03.280 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:08:03.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:08:03.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.285 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:08:03.290 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-26T02:08:03.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:08:03.292 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:08:03.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:08:03.294 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:08:03.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.298 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.299 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.301 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.302 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.307 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.312 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.315 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.316 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.317 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.319 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.320 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.323 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.326 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.327 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.328 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.331 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.332 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.333 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.335 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.336 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.337 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.338 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.342 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.347 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.348 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.350 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.353 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.354 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.358 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.359 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.360 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.361 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.362 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.364 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.365 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-26T02:08:03.367 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-26T02:08:03.368 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.370 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.371 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.373 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-26T02:08:03.375 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.379 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.380 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.383 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.388 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.391 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T02:08:03.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.393 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.394 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.395 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.396 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.398 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-26T02:08:03.401 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.403 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.404 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.406 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.407 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.410 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.411 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.415 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.416 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.417 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.425 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.428 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.431 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.433 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.435 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.439 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-26T02:08:03.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.444 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.446 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.447 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.448 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.455 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-26T02:08:03.457 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.463 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.467 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.468 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.470 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.472 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.474 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:08:03.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.476 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.478 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.480 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.482 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.483 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.484 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.485 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.487 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.492 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.494 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.495 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.499 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.502 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-26T02:08:03.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.507 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.508 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.512 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.513 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.514 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.515 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.517 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.519 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.521 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-04-26T02:08:03.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-04-26T02:08:03.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-04-26T02:08:03.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T02:08:03.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-04-26T02:08:03.534 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.539 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.540 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.542 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.544 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.548 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-04-26T02:08:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.552 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.555 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.556 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-04-26T02:08:03.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-04-26T02:08:03.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-04-26T02:08:03.559 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-04-26T02:08:03.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.561 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-04-26T02:08:03.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-04-26T02:08:03.564 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-04-26T02:08:03.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-04-26T02:08:03.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-04-26T02:08:03.567 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-04-26T02:08:03.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-04-26T02:08:03.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-04-26T02:08:03.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.571 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.580 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.582 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.586 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.587 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.591 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-04-26T02:08:03.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-04-26T02:08:03.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-04-26T02:08:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T02:08:03.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.608 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.616 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.622 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T02:08:03.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T02:08:03.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T02:08:03.626 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:03 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/444940646' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.628 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.631 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T02:08:03.632 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "[]", 2024-04-26T02:08:03.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "{}", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.643 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.655 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T02:08:03.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T02:08:03.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T02:08:03.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T02:08:03.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T02:08:03.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.662 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.663 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.664 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.665 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.666 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T02:08:03.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.668 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.669 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.672 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.673 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-04-26T02:08:03.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T02:08:03.682 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.684 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T02:08:03.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T02:08:03.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.695 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.700 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.702 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.705 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.707 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.708 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.709 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.712 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-04-26T02:08:03.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.720 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.723 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.727 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.728 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.730 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.736 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.740 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T02:08:03.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.747 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.749 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.751 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.752 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.754 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.755 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.756 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.759 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.760 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.762 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.763 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.764 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.766 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.769 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.770 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.772 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.774 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.775 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.776 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.778 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.782 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.783 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-04-26T02:08:03.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.790 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.792 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-04-26T02:08:03.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-04-26T02:08:03.795 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T02:08:03.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.799 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T02:08:03.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-04-26T02:08:03.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.811 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.812 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.815 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-04-26T02:08:03.816 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T02:08:03.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-04-26T02:08:03.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.819 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.822 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.828 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.829 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.833 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.834 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.837 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.838 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.839 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.845 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.849 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.852 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-04-26T02:08:03.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.856 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-04-26T02:08:03.857 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-04-26T02:08:03.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.862 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.863 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.864 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.865 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.869 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.870 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.871 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.872 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.877 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.879 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.881 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.882 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.883 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.884 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-04-26T02:08:03.885 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-04-26T02:08:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.887 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-04-26T02:08:03.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-04-26T02:08:03.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-04-26T02:08:03.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-04-26T02:08:03.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.894 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.895 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.897 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.902 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.903 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.904 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.905 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.907 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.908 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.909 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.911 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.915 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.916 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T02:08:03.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.919 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.922 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.923 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-04-26T02:08:03.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.925 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-04-26T02:08:03.928 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T02:08:03.929 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T02:08:03.930 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T02:08:03.931 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T02:08:03.932 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.933 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 92, 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1835495536 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2533185240 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T02:08:03.935 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4006708680 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1182533737 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 272756484 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-04-26T02:08:03.937 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2823668765 2024-04-26T02:08:03.938 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.938 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.938 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:03.938 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-04-26T02:08:03.938 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:08:03.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4cf40a3d90 msgr2=0x7f4cf40a6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:03.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4cf40a3d90 0x7f4cf40a6250 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cf8031440 tx=0x7f4cf8005a20 comp rx=0 tx=0).stop 2024-04-26T02:08:03.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 msgr2=0x7f4d101a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:03.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d101a05a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4d0000ea60 tx=0x7f4d0000ee60 comp rx=0 tx=0).stop 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 shutdown_connections 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4cf40a3d90 0x7f4cf40a6250 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d1019a6b0 0x7f4d1019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4d10109360 0x7f4d101a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 --2- 172.21.15.37:0/444940646 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4d10104290 0x7f4d101a05a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.675+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 >> 172.21.15.37:0/444940646 conn(0x7f4d100ffad0 msgr2=0x7f4d10100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.676+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 shutdown_connections 2024-04-26T02:08:03.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:02.676+0000 7f4d0fe00640 1 -- 172.21.15.37:0/444940646 wait complete. 2024-04-26T02:08:03.946 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T02:08:03.946 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T02:08:03.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:03 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/444940646' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:08:03.947 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:03 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/444940646' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T02:08:04.441 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:04.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:04 smithi037 ceph-mon[27906]: pgmap v312: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T02:08:04.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:04 smithi045 ceph-mon[32520]: pgmap v312: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T02:08:04.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:04 smithi071 ceph-mon[32578]: pgmap v312: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-26T02:08:06.454 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:06 smithi071 ceph-mon[32578]: pgmap v313: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-04-26T02:08:06.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:06 smithi037 ceph-mon[27906]: pgmap v313: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-04-26T02:08:06.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:06 smithi045 ceph-mon[32520]: pgmap v313: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 21 op/s 2024-04-26T02:08:07.182 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 2944, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.004976152, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000086841, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001375659, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:07.183 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.001557973, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000519324 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.184 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 241, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 230, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1594656, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 141984, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.109028578, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.038354656, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.083794957, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.010809690, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 229, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.026384170, 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000115214 2024-04-26T02:08:07.185 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1594656, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 141984 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 176, 2024-04-26T02:08:07.186 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 186960, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.035642380, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.025403591, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.004076101, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 173, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.027772975, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000160537 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:07.187 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 186960 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:07.188 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:07.189 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T02:08:07.190 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:07.191 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T02:08:07.192 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 243, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1560671, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 243, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 243, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1560671, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:07.193 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:07.194 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:08:07.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:07 smithi037 ceph-mon[27906]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:08:07.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:07 smithi045 ceph-mon[32520]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:08:07.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:07 smithi071 ceph-mon[32578]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T02:08:08.100 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-26T02:08:08.100 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-26T02:08:08.100 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-26T02:08:08.100 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T02:08:08.101 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T02:08:08.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:08 smithi037 ceph-mon[27906]: pgmap v315: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-04-26T02:08:08.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:08 smithi045 ceph-mon[32520]: pgmap v315: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-04-26T02:08:08.533 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:08.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:08 smithi071 ceph-mon[32578]: pgmap v315: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-04-26T02:08:10.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:10 smithi037 ceph-mon[27906]: pgmap v316: 97 pgs: 97 active+clean; 2.7 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-04-26T02:08:10.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:10 smithi037 ceph-mon[27906]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:08:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:10 smithi045 ceph-mon[32520]: pgmap v316: 97 pgs: 97 active+clean; 2.7 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-04-26T02:08:10.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:10 smithi045 ceph-mon[32520]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:08:10.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:10 smithi071 ceph-mon[32578]: pgmap v316: 97 pgs: 97 active+clean; 2.7 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 25 op/s 2024-04-26T02:08:10.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:10 smithi071 ceph-mon[32578]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T02:08:10.968 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:08:10.969 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:10.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 252, 2024-04-26T02:08:10.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 239, 2024-04-26T02:08:10.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1599504, 2024-04-26T02:08:10.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 146912, 2024-04-26T02:08:10.969 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:10.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:10.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.116417495, 2024-04-26T02:08:10.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.026278564, 2024-04-26T02:08:10.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.109932762, 2024-04-26T02:08:10.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.012626311, 2024-04-26T02:08:10.970 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:10.971 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 239, 2024-04-26T02:08:10.971 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.025164497, 2024-04-26T02:08:10.971 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000105290 2024-04-26T02:08:10.971 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.971 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1599504, 2024-04-26T02:08:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 146912 2024-04-26T02:08:10.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.973 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:10.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-04-26T02:08:10.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 182, 2024-04-26T02:08:10.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:08:10.973 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 188016, 2024-04-26T02:08:10.974 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:10.974 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:10.974 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.038516576, 2024-04-26T02:08:10.974 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.026761136, 2024-04-26T02:08:10.974 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.006052009, 2024-04-26T02:08:10.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:10.975 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:10.975 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 179, 2024-04-26T02:08:10.975 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.044845743, 2024-04-26T02:08:10.975 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000250534 2024-04-26T02:08:10.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:10.976 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:10.976 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:10.976 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:10.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.976 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:08:10.977 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 188016 2024-04-26T02:08:10.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.977 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:10.977 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:10.977 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:10.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:10.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:10.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:10.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:10.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001098708, 2024-04-26T02:08:10.978 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:10.979 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000944650, 2024-04-26T02:08:10.979 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:10.979 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:10.979 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:10.979 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:10.979 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:10.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.980 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:10.980 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:10.980 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:10.980 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:10.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.981 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:10.981 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:10.981 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.981 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T02:08:10.981 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T02:08:10.981 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T02:08:10.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.982 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T02:08:10.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:10.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:10.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:10.982 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:10.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:10.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:10.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:10.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:10.983 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:10.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:10.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:10.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:10.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:10.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:10.984 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:10.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:10.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:10.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:10.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:10.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:10.985 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:10.986 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:10.986 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:10.986 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:10.986 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:10.986 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:10.986 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:10.987 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:10.987 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:10.987 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:10.987 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:10.987 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T02:08:10.987 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-26T02:08:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:10.988 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:10.989 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:10.989 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T02:08:10.989 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T02:08:10.989 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:10.989 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:10.989 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:10.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:10.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:10.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T02:08:10.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:10.990 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:10.990 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:10.991 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T02:08:10.991 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:10.991 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T02:08:10.991 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:10.991 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T02:08:10.992 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:10.992 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T02:08:10.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:10.992 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:10.992 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T02:08:10.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T02:08:10.993 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T02:08:10.993 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 250, 2024-04-26T02:08:10.993 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1561581, 2024-04-26T02:08:10.993 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:10.993 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 250, 2024-04-26T02:08:10.993 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T02:08:10.994 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T02:08:10.994 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 250, 2024-04-26T02:08:10.994 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1561581, 2024-04-26T02:08:10.994 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T02:08:10.994 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:10.995 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:10.995 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:10.995 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:10.995 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:10.995 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:08:11.815 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T02:08:11.815 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T02:08:11.816 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T02:08:11.816 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T02:08:11.816 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T02:08:12.282 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:12.495 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:12 smithi037 ceph-mon[27906]: pgmap v318: 97 pgs: 97 active+clean; 2.7 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:08:12.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:12 smithi045 ceph-mon[32520]: pgmap v318: 97 pgs: 97 active+clean; 2.7 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:08:12.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:12 smithi071 ceph-mon[32578]: pgmap v318: 97 pgs: 97 active+clean; 2.7 MiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 127 B/s wr, 1 op/s 2024-04-26T02:08:14.421 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:14 smithi037 ceph-mon[27906]: pgmap v319: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:14.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:14 smithi045 ceph-mon[32520]: pgmap v319: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:14.639 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 244, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 233, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1595472, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 143904, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.101450546, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.027071377, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.081714044, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.011215532, 2024-04-26T02:08:14.640 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 232, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.025584756, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000110279 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1595472, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 143904 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:14.641 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 181, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3264, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 187936, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.040023650, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.027475802, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.006676788, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 179, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.033539238, 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000187370 2024-04-26T02:08:14.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 187936 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:14.643 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.000922571, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000738389, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:14.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:14.645 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:14.646 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:14.647 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T02:08:14.648 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 246, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1561061, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:14.649 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 246, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 246, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1561061, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:14.650 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:08:14.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:14 smithi071 ceph-mon[32578]: pgmap v319: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:15.445 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-26T02:08:15.445 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-26T02:08:15.445 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-26T02:08:15.445 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T02:08:15.445 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T02:08:15.909 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:16.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:16 smithi037 ceph-mon[27906]: pgmap v320: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 234 B/s wr, 2 op/s 2024-04-26T02:08:16.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:16 smithi045 ceph-mon[32520]: pgmap v320: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 234 B/s wr, 2 op/s 2024-04-26T02:08:16.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:16 smithi071 ceph-mon[32578]: pgmap v320: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 234 B/s wr, 2 op/s 2024-04-26T02:08:18.332 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:18 smithi037 ceph-mon[27906]: pgmap v321: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:08:18.337 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-04-26T02:08:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:18.337 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.004390612, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001218240, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:18.338 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 252, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 241, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1598272, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 148720, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.119757246, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.027836363, 2024-04-26T02:08:18.339 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.121509741, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.011226961, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 240, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.026739781, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000111415 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1598272, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 148720 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:18.340 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 186, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3264, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 188816, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.038032284, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.028264204, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.004320931, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 184, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.029353430, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000159529 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.341 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 188816 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:18.342 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:18.343 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:18.344 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148714, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-04-26T02:08:18.345 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-04-26T02:08:18.346 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 254, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1562738, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 254, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 254, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1562738, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-04-26T02:08:18.347 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-04-26T02:08:18.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:18 smithi045 ceph-mon[32520]: pgmap v321: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:08:18.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:18 smithi071 ceph-mon[32578]: pgmap v321: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 204 B/s wr, 2 op/s 2024-04-26T02:08:19.146 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T02:08:19.147 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T02:08:19.147 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T02:08:19.147 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T02:08:19.147 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T02:08:19.408 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:08:20.482 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:20 smithi045 ceph-mon[32520]: pgmap v322: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:08:20.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:20 smithi071 ceph-mon[32578]: pgmap v322: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:08:20.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:20 smithi037 ceph-mon[27906]: pgmap v322: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 102 B/s wr, 2 op/s 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1782, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1782, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2081496, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 565862, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.159015433, 2024-04-26T02:08:21.077 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.060966160, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.037923734, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.044537864, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1781, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.065371909, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000036705 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.078 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 7, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 184, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2136, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 552282, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 8, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 5, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.011517772, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.007053063, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.002236220, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000119747, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 180, 2024-04-26T02:08:21.079 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.034636669, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000192425 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 550944 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 1085, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 1073, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1817777, 2024-04-26T02:08:21.080 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 408531, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 3, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 2, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.118136587, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.035215250, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.049492921, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.034017807, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 1072, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.037304935, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000034799 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.081 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1615248, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 146816 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 1, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-MDSRank": { 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 4373, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.111155419, 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000025418 2024-04-26T02:08:21.082 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "finisher-PurgeQueue": { 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "queue_len": 0, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "complete_latency": { 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "mds": { 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "request": 0, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "reply": 0, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "reply_latency": { 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.083 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_reply": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "forward": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_complete": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_fetch_keys": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_commit": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_split": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_merge": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes": 30, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_top": 10, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_bottom": 20, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pin_tail": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_pinned": 10, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_expired": 0, 2024-04-26T02:08:21.084 INFO:teuthology.orchestra.run.smithi045.stdout: "inodes_with_caps": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "caps": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "subtrees": 2, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_hit": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_forward": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_discover": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_remote_ino": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "traverse_lock": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "load_cent": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "q": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "exported": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "exported_inodes": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "imported": 0, 2024-04-26T02:08:21.085 INFO:teuthology.orchestra.run.smithi045.stdout: "imported_inodes": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_dir_fetch": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "openino_peer_discover": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rfiles": 4, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rbytes": 234, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "root_rsnaps": 7, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_set_tag": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "scrub_file_inodes": 0, 2024-04-26T02:08:21.086 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_grant": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_caps_dirty": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_cap_release": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "process_request_cap_release": 0 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_cache": { 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_update_receipt": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_try_discover": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_send_discover": 0, 2024-04-26T02:08:21.087 INFO:teuthology.orchestra.run.smithi045.stdout: "dir_handle_discover": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_delayed": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_created": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_enqueued": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_reintegrated": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "strays_migrated": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_processing": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_started": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "recovery_completed": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_path": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_quiesce_inode": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_exportdir": 0, 2024-04-26T02:08:21.088 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_flush": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_fragstats": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "ireq_inodestats": 0 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_log": { 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "evlrg": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "evadd": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "evex": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "evtrm": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "ev": 314, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "evexg": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "evexd": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "segadd": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "segex": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "segtrm": 0, 2024-04-26T02:08:21.089 INFO:teuthology.orchestra.run.smithi045.stdout: "seg": 2, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "segmjr": 24, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "segexg": 0, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "segexd": 0, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "expos": 5710607, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "wrpos": 5746761, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "rdpos": 5746649, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "jlat": { 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "replayed": 314 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.090 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_mem": { 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "ino": 21, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "ino+": 24, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "ino-": 3, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "dir": 13, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "dir+": 13, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "dir-": 0, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "dn": 30, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "dn+": 30, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "dn-": 0, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "cap": 0, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "cap+": 0, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "cap-": 0, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "rss": 50444, 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: "heap": 215416 2024-04-26T02:08:21.091 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_server": { 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_client_request": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "dispatch_server_request": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_request": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_client_session": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "handle_peer_request": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "req_create_latency": { 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getattr_latency": { 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.092 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getfilelock_latency": { 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "req_link_latency": { 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookup_latency": { 2024-04-26T02:08:21.093 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookuphash_latency": { 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupino_latency": { 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.094 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupname_latency": { 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupparent_latency": { 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lookupsnap_latency": { 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.095 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "req_lssnap_latency": { 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mkdir_latency": { 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.096 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mknod_latency": { 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "req_mksnap_latency": { 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "req_open_latency": { 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.097 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "req_readdir_latency": { 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rename_latency": { 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "req_renamesnap_latency": { 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.098 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "req_snapdiff_latency": { 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmdir_latency": { 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmsnap_latency": { 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.099 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "req_rmxattr_latency": { 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setattr_latency": { 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setdirlayout_latency": { 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.100 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setfilelock_latency": { 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setlayout_latency": { 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "req_setxattr_latency": { 2024-04-26T02:08:21.101 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "req_symlink_latency": { 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "req_unlink_latency": { 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.102 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "req_getvxattr_latency": { 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_sessions": { 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "session_count": 2, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "session_add": 5, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "session_remove": 3, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_open": 4, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "sessions_stale": 0, 2024-04-26T02:08:21.103 INFO:teuthology.orchestra.run.smithi045.stdout: "total_load": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "average_load": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "avg_session_uptime": 3798, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "mdthresh_evicted": 0 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:21.104 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:21.105 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 190622, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 65, 2024-04-26T02:08:21.106 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 8032, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 167, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 110888, 2024-04-26T02:08:21.107 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 752, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "objecter": { 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_active": 0, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_laggy": 0, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send": 2605, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_send_bytes": 26, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_resend": 4, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_reply": 2605, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "op_latency": { 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2605, 2024-04-26T02:08:21.108 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 1.072982448, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000411893 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "op_inflight": 0, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "oplen_avg": { 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 2605, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 2606 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "op": 2605, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "op_r": 2605, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "op_w": 0, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "op_rmw": 0, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "op_pg": 0, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_stat": 1653, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_create": 0, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_read": 949, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_write": 0, 2024-04-26T02:08:21.109 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writefull": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_writesame": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_append": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_zero": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_truncate": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_delete": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_mapext": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_sparse_read": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_clonerange": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_getxattr": 2, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_setxattr": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_rmxattr": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_call": 0, 2024-04-26T02:08:21.110 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_watch": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_notify": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "osdop_other": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_active": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_send": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_resend": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "linger_ping": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_active": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_send": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "poolop_resend": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_active": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_send": 0, 2024-04-26T02:08:21.111 INFO:teuthology.orchestra.run.smithi045.stdout: "poolstat_resend": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_active": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_send": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "statfs_resend": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "command_active": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "command_send": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "command_resend": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "map_epoch": 100, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "map_full": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "map_inc": 11, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_sessions": 5, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_open": 5, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_session_close": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_laggy": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_wr": 0, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_rd": 2, 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_del": 0 2024-04-26T02:08:21.112 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "oft": { 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_objs": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_updates": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "omap_total_removes": 0 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "osd-slow-ops": { 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "slow_ops_count": 0 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "purge_queue": { 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executing_high_water": 0, 2024-04-26T02:08:21.113 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed_ops": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_executed": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "pq_item_in_journal": 0 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 2870, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 3626714, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 2870, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-26T02:08:21.114 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2870, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 3626714, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_bytes": { 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:21.115 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 2605, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 1552467, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2605, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1552467, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-objecter_ops": { 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 1024, 2024-04-26T02:08:21.116 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 2605, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 2605, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 2605, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 2605, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.117 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.118 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-write_buf_throttle-0x565312c446e0": { 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 3758096384, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 0, 2024-04-26T02:08:21.119 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.120 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:21.120 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-26T02:08:21.801 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-04-26T02:08:21.801 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-04-26T02:08:21.801 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T02:08:21.802 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T02:08:21.802 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T02:08:22.053 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:08:22.435 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:22 smithi045 ceph-mon[32520]: pgmap v323: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 B/s rd, 92 B/s wr, 1 op/s 2024-04-26T02:08:22.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:22 smithi071 ceph-mon[32578]: pgmap v323: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 B/s rd, 92 B/s wr, 1 op/s 2024-04-26T02:08:22.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:22 smithi037 ceph-mon[27906]: pgmap v323: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 B/s rd, 92 B/s wr, 1 op/s 2024-04-26T02:08:23.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:23 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:08:23.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:23 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:23.464 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:23 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:23.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:23 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:08:23.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:23 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:23.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:23 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:23.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:23 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T02:08:23.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:23 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:23.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:23 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:23.841 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-26T02:08:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:23.841 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001285165, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000307041, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:23.842 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 254, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 241, 2024-04-26T02:08:23.843 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1600048, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 148192, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.049316611, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015618687, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.034244471, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.006490524, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 241, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.012481606, 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000051790 2024-04-26T02:08:23.844 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1600048, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 148192 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 184, 2024-04-26T02:08:23.845 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 188368, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.012280132, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008476074, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001232177, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 181, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.027141797, 2024-04-26T02:08:23.846 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000149954 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 188368 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-26T02:08:23.847 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:23.848 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:23.849 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:23.850 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148692, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-26T02:08:23.851 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:23.852 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 252, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1561841, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:23.853 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 252, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 252, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1561841, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:23.854 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-26T02:08:24.443 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:24 smithi045 ceph-mon[32520]: pgmap v324: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:08:24.443 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:24 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:24.443 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:24 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:24.471 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-04-26T02:08:24.471 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-04-26T02:08:24.472 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-04-26T02:08:24.472 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T02:08:24.472 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T02:08:24.652 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:24 smithi071 ceph-mon[32578]: pgmap v324: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:08:24.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:24 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:24.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:24 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:24.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:24 smithi037 ceph-mon[27906]: pgmap v324: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-04-26T02:08:24.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:24 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:24.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:24 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.256 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:08:25.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.653 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: pgmap v325: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: Unable to set osd_memory_target on smithi071 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.654 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:25 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: pgmap v325: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.903 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-26T02:08:25.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: Unable to set osd_memory_target on smithi071 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-26T02:08:25.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.904 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:25 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: pgmap v325: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi071 to 176.2M 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: Unable to set osd_memory_target on smithi071 to 184769536: error parsing value: Value '184769536' is below minimum 939524096 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:25.960 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:25 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 253, 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 240, 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1599776, 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 147552, 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:26.795 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.050719023, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015034250, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.036093488, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.006435019, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 240, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.011767695, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000049032 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:26.796 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1599776, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 147552 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 3, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 184, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 480, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 266448, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 6, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.012862887, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008571294, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001658519, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:26.797 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 180, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.034893178, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000193850 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 266448 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:26.798 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.000303160, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000229031, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:26.799 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.800 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:26.801 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:26.802 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148732, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-26T02:08:26.803 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-26T02:08:26.804 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 252, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1561738, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 252, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 252, 2024-04-26T02:08:26.805 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1561738, 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:26.806 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:26 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.174 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.175 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:26 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:26 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:27.480 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.h 2024-04-26T02:08:27.480 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.h 2024-04-26T02:08:27.480 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.h 2024-04-26T02:08:27.480 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T02:08:27.480 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T02:08:27.749 INFO:teuthology.orchestra.run.smithi045.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T02:08:28.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:27 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi037 to 177.5M 2024-04-26T02:08:28.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:27 smithi071 ceph-mon[32578]: Unable to set osd_memory_target on smithi037 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T02:08:28.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:27 smithi071 ceph-mon[32578]: pgmap v326: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T02:08:28.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:27 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:28.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:27 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:28.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:27 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi037 to 177.5M 2024-04-26T02:08:28.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:27 smithi037 ceph-mon[27906]: Unable to set osd_memory_target on smithi037 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T02:08:28.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:27 smithi037 ceph-mon[27906]: pgmap v326: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T02:08:28.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:27 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:28.164 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:27 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:28.211 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:27 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi037 to 177.5M 2024-04-26T02:08:28.212 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:27 smithi045 ceph-mon[32520]: Unable to set osd_memory_target on smithi037 to 186173030: error parsing value: Value '186173030' is below minimum 939524096 2024-04-26T02:08:28.212 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:27 smithi045 ceph-mon[32520]: pgmap v326: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-04-26T02:08:28.212 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:27 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:28.212 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:27 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:29.067 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:08:29.068 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:28 smithi045 ceph-mon[32520]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:08:29.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:28 smithi071 ceph-mon[32578]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T02:08:29.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T02:08:29.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T02:08:29.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:28 smithi037 ceph-mon[27906]: from='mgr.14856 ' entity='mgr.x' 2024-04-26T02:08:29.296 INFO:teuthology.orchestra.run.smithi045.stdout:{ 2024-04-26T02:08:29.297 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:29.297 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 4, 2024-04-26T02:08:29.297 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 4, 2024-04-26T02:08:29.297 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 2944, 2024-04-26T02:08:29.297 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 448, 2024-04-26T02:08:29.298 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:29.298 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:29.298 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.001625265, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.000040655, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.000543890, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 3, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000757937, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000252645 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.299 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 2, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 188, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 189072, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.012277876, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.008463258, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.001249653, 2024-04-26T02:08:29.300 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 185, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.032588547, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000176154 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 189072 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.301 INFO:teuthology.orchestra.run.smithi045.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_messages": 256, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages": 245, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_bytes": 1599360, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_bytes": 151280, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_total_time": 0.050780412, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_send_time": 0.015855214, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_recv_time": 0.037089096, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_running_fast_dispatch_time": 0.006243161, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 244, 2024-04-26T02:08:29.302 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.013450978, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000055126 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_recv_encrypted_bytes": 1599360, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "msgr_send_encrypted_bytes": 151280 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "cct": { 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "total_workers": 0, 2024-04-26T02:08:29.303 INFO:teuthology.orchestra.run.smithi045.stdout: "unhealthy_workers": 0 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "mempool": { 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:29.304 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:29.305 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_items": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_bytes": 148754, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_anon_items": 31, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_bytes": 0, 2024-04-26T02:08:29.306 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_items": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_items": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "pgmap_items": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "mds_co_items": 2, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:29.307 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_1_items": 0, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "unittest_2_items": 0 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: }, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "val": 0, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "max": 104857600, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "get_started": 0, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "get": 258, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "get_sum": 1563258, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "get_or_fail_success": 258, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "take": 0, 2024-04-26T02:08:29.308 INFO:teuthology.orchestra.run.smithi045.stdout: "take_sum": 0, 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: "put": 258, 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: "put_sum": 1563258, 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: "wait": { 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: "avgcount": 0, 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: "sum": 0.000000000, 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: "avgtime": 0.000000000 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout: } 2024-04-26T02:08:29.309 INFO:teuthology.orchestra.run.smithi045.stdout:} 2024-04-26T02:08:29.929 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-26T02:08:29.930 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-26T02:08:29.930 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-26T02:08:29.930 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T02:08:29.930 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T02:08:30.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:29 smithi071 ceph-mon[32578]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-26T02:08:30.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:29 smithi071 ceph-mon[32578]: Unable to set osd_memory_target on smithi045 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T02:08:30.153 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:29 smithi071 ceph-mon[32578]: pgmap v327: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:30.176 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:08:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:29 smithi037 ceph-mon[27906]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-26T02:08:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:29 smithi037 ceph-mon[27906]: Unable to set osd_memory_target on smithi045 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T02:08:30.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:29 smithi037 ceph-mon[27906]: pgmap v327: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:30.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:29 smithi045 ceph-mon[32520]: Adjusting osd_memory_target on smithi045 to 177.2M 2024-04-26T02:08:30.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:29 smithi045 ceph-mon[32520]: Unable to set osd_memory_target on smithi045 to 185896345: error parsing value: Value '185896345' is below minimum 939524096 2024-04-26T02:08:30.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:29 smithi045 ceph-mon[32520]: pgmap v327: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 252, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 241, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1598272, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 148720, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.050938382, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.014726440, 2024-04-26T02:08:31.982 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.037718210, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.006063575, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 240, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.013958719, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000058161 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1598272, 2024-04-26T02:08:31.983 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 148720 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 6, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 187, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 3264, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 267248, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.012121111, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.008038746, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001688269, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:31.984 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 185, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.027884782, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000150728 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 267248 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:31.985 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.000165525, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000107924, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:31.986 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:31.987 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:31.988 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:31.989 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148670, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:31.990 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-26T02:08:31.991 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 254, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1562738, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 254, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 254, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1562738, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:31.992 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-26T02:08:32.152 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:32 smithi071 ceph-mon[32578]: pgmap v328: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:32.255 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:32 smithi045 ceph-mon[32520]: pgmap v328: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:32.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:32 smithi037 ceph-mon[27906]: pgmap v328: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:32.695 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-04-26T02:08:32.695 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-04-26T02:08:32.696 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-04-26T02:08:32.696 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T02:08:32.696 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T02:08:32.934 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:08:34.099 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:34 smithi071 ceph-mon[32578]: pgmap v329: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:34.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:34 smithi037 ceph-mon[27906]: pgmap v329: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:34.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:34 smithi045 ceph-mon[32520]: pgmap v329: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:34.706 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.000967072, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000119887, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.707 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 6, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 187, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 3264, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 188992, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 5, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.011316695, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.007604884, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001434000, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:34.708 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 185, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.032289881, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000174539 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 188992 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 253, 2024-04-26T02:08:34.709 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 242, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1598544, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 149360, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.051778228, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.016178538, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.037392463, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.006673474, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 241, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.012929749, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000053650 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:34.710 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1598544, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 149360 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:34.711 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:34.712 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-26T02:08:34.713 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148670, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 30, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:34.714 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:34.715 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 255, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1562868, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 255, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 255, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1562868, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:34.716 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:34.717 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:34.717 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-26T02:08:35.424 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-04-26T02:08:35.424 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-04-26T02:08:35.424 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-04-26T02:08:35.424 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T02:08:35.425 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T02:08:35.661 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:08:36.059 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:36 smithi071 ceph-mon[32578]: pgmap v330: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:36.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:36 smithi037 ceph-mon[27906]: pgmap v330: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:36.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:36 smithi045 ceph-mon[32520]: pgmap v330: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 3129, 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2523, 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 938080, 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 2530478, 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 10, 2024-04-26T02:08:37.404 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.251433891, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.067182594, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.076865539, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.091536591, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2522, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.164920107, 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000065392 2024-04-26T02:08:37.405 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 918, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000155916, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000169 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 340560, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 102048 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2486, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2390, 2024-04-26T02:08:37.406 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 2199222, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 1640951, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 67, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.203056026, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.053341258, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.094492456, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.055335979, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2389, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.243511000, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000101930 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:37.407 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 170, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000024988, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000146 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1672112, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 162880 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2176, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 1660, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 722247, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 1169530, 2024-04-26T02:08:37.408 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 15, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 5, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.209305954, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.055318366, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.072260048, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.076175527, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1657, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.245492489, 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000148154 2024-04-26T02:08:37.409 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1341, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000268739, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000200 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 104624, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 520320 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 1, 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-26T02:08:37.410 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-MDSRank": { 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 916, 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.159885966, 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000174547 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "finisher-PurgeQueue": { 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "queue_len": 0, 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "complete_latency": { 2024-04-26T02:08:37.411 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4210, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.114395419, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000027172 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "mds": { 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "request": 185, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "reply": 185, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "reply_latency": { 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 185, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 10.364709166, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.056025454 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.412 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_reply": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "forward": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_complete": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_fetch_keys": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_commit": 123, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_split": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_merge": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes": 35, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_top": 9, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_bottom": 26, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pin_tail": 0, 2024-04-26T02:08:37.413 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_pinned": 17, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_expired": 0, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "inodes_with_caps": 3, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "caps": 4, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "subtrees": 2, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse": 236, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_hit": 227, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_forward": 0, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_discover": 0, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_dir_fetch": 0, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_remote_ino": 0, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "traverse_lock": 0, 2024-04-26T02:08:37.414 INFO:teuthology.orchestra.run.smithi071.stdout: "load_cent": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "q": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "exported": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "exported_inodes": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "imported": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "imported_inodes": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_dir_fetch": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_backtrace_fetch": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "openino_peer_discover": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rfiles": 4, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rbytes": 234, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "root_rsnaps": 7, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_set_tag": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_inotable_repaired": 0, 2024-04-26T02:08:37.415 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_inodes": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "scrub_file_inodes": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_inode_file_caps": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_revoke": 32, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_grant": 104, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps": 173, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_caps_dirty": 116, 2024-04-26T02:08:37.416 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_cap_release": 6, 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "process_request_cap_release": 27 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_cache": { 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update": 0, 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_update_receipt": 0, 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_try_discover": 0, 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_send_discover": 0, 2024-04-26T02:08:37.417 INFO:teuthology.orchestra.run.smithi071.stdout: "dir_handle_discover": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays": 2, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_delayed": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "num_strays_enqueuing": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_created": 5, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_enqueued": 3, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_reintegrated": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "strays_migrated": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_processing": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_enqueued": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "num_recovering_prioritized": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_started": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "recovery_completed": 0, 2024-04-26T02:08:37.418 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_path": 3, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_quiesce_inode": 40, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_exportdir": 0, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_flush": 0, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragmentdir": 0, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_fragstats": 0, 2024-04-26T02:08:37.419 INFO:teuthology.orchestra.run.smithi071.stdout: "ireq_inodestats": 0 2024-04-26T02:08:37.421 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.421 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_log": { 2024-04-26T02:08:37.421 INFO:teuthology.orchestra.run.smithi071.stdout: "evlrg": 0, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "evadd": 314, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "evex": 265, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "evtrm": 265, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "ev": 49, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "evexg": 0, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "evexd": 6, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "segadd": 24, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "segex": 23, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "segtrm": 23, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "seg": 1, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "segmjr": 1, 2024-04-26T02:08:37.422 INFO:teuthology.orchestra.run.smithi071.stdout: "segexg": 0, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "segexd": 1, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "expos": 5710607, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "wrpos": 5746761, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "rdpos": 0, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "jlat": { 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 274, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.737221512, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002690589 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "replayed": 0 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_mem": { 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "ino": 28, 2024-04-26T02:08:37.423 INFO:teuthology.orchestra.run.smithi071.stdout: "ino+": 31, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "ino-": 3, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "dir": 20, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "dir+": 20, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "dir-": 0, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "dn": 35, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "dn+": 40, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "dn-": 5, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "cap": 4, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "cap+": 26, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "cap-": 22, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "rss": 54448, 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: "heap": 231800 2024-04-26T02:08:37.424 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_server": { 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_client_request": 214, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "dispatch_server_request": 0, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_request": 185, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_client_session": 35, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "handle_peer_request": 0, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "req_create_latency": { 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.007350333, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001050047 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.425 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getattr_latency": { 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 116, 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.025663333, 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000221235 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getfilelock_latency": { 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.426 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "req_link_latency": { 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookup_latency": { 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002327055, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000332436 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookuphash_latency": { 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.427 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupino_latency": { 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000291042, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000145521 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupname_latency": { 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.428 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupparent_latency": { 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lookupsnap_latency": { 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "req_lssnap_latency": { 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 11, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002435929, 2024-04-26T02:08:37.429 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000221448 2024-04-26T02:08:37.430 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.430 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mkdir_latency": { 2024-04-26T02:08:37.430 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 8, 2024-04-26T02:08:37.430 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 10.209904867, 2024-04-26T02:08:37.430 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 1.276238108 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mknod_latency": { 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "req_mksnap_latency": { 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 11, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.050998108, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004636191 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "req_open_latency": { 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-26T02:08:37.431 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.001017353, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000508676 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_readdir_latency": { 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.004377294, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.002188647 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rename_latency": { 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.002467672, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.001233836 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "req_renamesnap_latency": { 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.432 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "req_snapdiff_latency": { 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmdir_latency": { 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 2, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.007478177, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003739088 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmsnap_latency": { 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 4, 2024-04-26T02:08:37.433 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.017880887, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.004470221 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "req_rmxattr_latency": { 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setattr_latency": { 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 1, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000931415, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000931415 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setdirlayout_latency": { 2024-04-26T02:08:37.434 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setfilelock_latency": { 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setlayout_latency": { 2024-04-26T02:08:37.435 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "req_setxattr_latency": { 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 7, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.022567452, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003223921 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "req_symlink_latency": { 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "req_unlink_latency": { 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 3, 2024-04-26T02:08:37.436 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.009018249, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.003006083 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_revoke_eviction": 0, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "cap_acquisition_throttle": 0, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "req_getvxattr_latency": { 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_sessions": { 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "session_count": 2, 2024-04-26T02:08:37.437 INFO:teuthology.orchestra.run.smithi071.stdout: "session_add": 5, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "session_remove": 3, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_open": 2, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "sessions_stale": 0, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "total_load": 29, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "average_load": 14, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "avg_session_uptime": 638, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "mdthresh_evicted": 0 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:37.438 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:37.439 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:37.440 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 319323, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 105, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:37.441 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 8032, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 167, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 139167, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 1078, 2024-04-26T02:08:37.442 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "objecter": { 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_active": 0, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_laggy": 0, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send": 5466, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_send_bytes": 1840929, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_resend": 31, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_reply": 5466, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "op_latency": { 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5466, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 50.855102602, 2024-04-26T02:08:37.443 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.009303897 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "op_inflight": 0, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "oplen_avg": { 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 5466, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 5788 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "op": 5466, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "op_r": 3, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "op_w": 5463, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "op_rmw": 0, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "op_pg": 0, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_stat": 103, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_create": 33, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_read": 3, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_write": 277, 2024-04-26T02:08:37.444 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writefull": 96, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_writesame": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_append": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_zero": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_truncate": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_delete": 4865, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_mapext": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_sparse_read": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_clonerange": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_getxattr": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_setxattr": 66, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_cmpxattr": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_rmxattr": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_resetxattrs": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_call": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_watch": 0, 2024-04-26T02:08:37.445 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_notify": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_pgls_filter": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "osdop_other": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_active": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_send": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_resend": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "linger_ping": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_active": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_send": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "poolop_resend": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_active": 0, 2024-04-26T02:08:37.446 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_send": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "poolstat_resend": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_active": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_send": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "statfs_resend": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "command_active": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "command_send": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "command_resend": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "map_epoch": 100, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "map_full": 0, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "map_inc": 11, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_sessions": 12, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_open": 12, 2024-04-26T02:08:37.447 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_session_close": 0, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_laggy": 0, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_wr": 323, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_rd": 0, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_del": 22 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "oft": { 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_objs": 1, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_kv_pairs": 0, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_updates": 0, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "omap_total_removes": 0 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "osd-slow-ops": { 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "slow_ops_count": 44 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.448 INFO:teuthology.orchestra.run.smithi071.stdout: "purge_queue": { 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops": 0, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_ops_high_water": 3584, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing": 0, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executing_high_water": 1, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed_ops": 4353, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_executed": 3, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "pq_item_in_journal": 0 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 7787, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 3089171, 2024-04-26T02:08:37.449 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 7787, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 7787, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 3089171, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.450 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_bytes": { 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 5466, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 1841256, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5466, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1841256, 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:37.451 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-objecter_ops": { 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 1024, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 0, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 5466, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 5466, 2024-04-26T02:08:37.452 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 5466, 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 5466, 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle": { 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:37.453 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 3, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 327, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 3, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 3, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 327, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.454 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-write_buf_throttle-0x558653e4e460": { 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 3758096384, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 314, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1552457, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 314, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-26T02:08:37.455 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 274, 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1552457, 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:37.456 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-26T02:08:38.232 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=7 2024-04-26T02:08:38.232 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=11 2024-04-26T02:08:38.232 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=4 2024-04-26T02:08:38.232 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T02:08:38.232 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T02:08:38.402 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:38 smithi071 ceph-mon[32578]: pgmap v331: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:38.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:38 smithi037 ceph-mon[27906]: pgmap v331: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:38.470 INFO:teuthology.orchestra.run.smithi071.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T02:08:38.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:38 smithi045 ceph-mon[32520]: pgmap v331: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:40.143 INFO:teuthology.orchestra.run.smithi071.stdout:{ 2024-04-26T02:08:40.143 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T02:08:40.143 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 0, 2024-04-26T02:08:40.143 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 2, 2024-04-26T02:08:40.143 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 0, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 0, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 4, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 0, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.001143177, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.000281654, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.144 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 2, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 191, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 320, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 267856, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 6, 2024-04-26T02:08:40.145 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 3, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.012560844, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.008495979, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.001406794, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 188, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.018587248, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000098868 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:40.146 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 267856 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_messages": 263, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages": 250, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_bytes": 1603120, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_bytes": 153632, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_created_connections": 2, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_active_connections": 1, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_total_time": 0.051366972, 2024-04-26T02:08:40.147 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_send_time": 0.016268154, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_recv_time": 0.035759157, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_running_fast_dispatch_time": 0.006297859, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 250, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.014324178, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000057296 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_handle_ack_lat": { 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.148 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_recv_encrypted_bytes": 1603120, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "msgr_send_encrypted_bytes": 153632 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "cct": { 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "total_workers": 0, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "unhealthy_workers": 0 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "mempool": { 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_bytes": 0, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "bloom_filter_items": 0, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_alloc_items": 0, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T02:08:40.149 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_data_items": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_other_items": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_bytes": 0, 2024-04-26T02:08:40.150 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_extent_items": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_bytes": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_blob_items": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T02:08:40.151 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_fsck_items": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_bytes": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_txc_items": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_bytes": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluestore_writing_items": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_bytes": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_items": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T02:08:40.152 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_reader_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "bluefs_file_writer_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_bytes": 148731, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_anon_items": 31, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_bytes": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "buffer_meta_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_bytes": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_bytes": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_mapbl_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_bytes": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osd_pglog_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_bytes": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_items": 0, 2024-04-26T02:08:40.153 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "osdmap_mapping_items": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_bytes": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "pgmap_items": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_bytes": 928, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "mds_co_items": 2, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_bytes": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_1_items": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_bytes": 0, 2024-04-26T02:08:40.154 INFO:teuthology.orchestra.run.smithi071.stdout: "unittest_2_items": 0 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: }, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "val": 0, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "max": 104857600, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "get_started": 0, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "get": 261, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "get_sum": 1563648, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_fail": 0, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "get_or_fail_success": 261, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "take": 0, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "take_sum": 0, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "put": 261, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "put_sum": 1563648, 2024-04-26T02:08:40.155 INFO:teuthology.orchestra.run.smithi071.stdout: "wait": { 2024-04-26T02:08:40.156 INFO:teuthology.orchestra.run.smithi071.stdout: "avgcount": 0, 2024-04-26T02:08:40.156 INFO:teuthology.orchestra.run.smithi071.stdout: "sum": 0.000000000, 2024-04-26T02:08:40.156 INFO:teuthology.orchestra.run.smithi071.stdout: "avgtime": 0.000000000 2024-04-26T02:08:40.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:40.156 INFO:teuthology.orchestra.run.smithi071.stdout: } 2024-04-26T02:08:40.156 INFO:teuthology.orchestra.run.smithi071.stdout:} 2024-04-26T02:08:40.156 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:40 smithi071 ceph-mon[32578]: pgmap v332: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:40.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:40 smithi037 ceph-mon[27906]: pgmap v332: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:40.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:40 smithi045 ceph-mon[32520]: pgmap v332: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:40.803 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-26T02:08:40.803 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-26T02:08:40.803 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-26T02:08:40.804 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T02:08:40.815 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T02:08:40.815 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T02:08:40.815 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:08:40.878 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-04-26T02:08:40.878 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T02:08:40.932 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-26T02:08:41.402 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:42.186 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:42 smithi037 ceph-mon[27906]: pgmap v333: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:42.307 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:42 smithi045 ceph-mon[32520]: pgmap v333: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:42.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:42 smithi071 ceph-mon[32578]: pgmap v333: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:43.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 -- 172.21.15.37:0/4186307318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 msgr2=0x7f75d81096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:43.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 --2- 172.21.15.37:0/4186307318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 0x7f75d81096b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f75cc0099c0 tx=0x7f75cc009cd0 comp rx=0 tx=0).stop 2024-04-26T02:08:43.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 -- 172.21.15.37:0/4186307318 shutdown_connections 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 --2- 172.21.15.37:0/4186307318 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 0x7f75d81096b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 --2- 172.21.15.37:0/4186307318 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d810e3b0 0x7f75d8115f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 --2- 172.21.15.37:0/4186307318 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d8109c80 0x7f75d810dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 -- 172.21.15.37:0/4186307318 >> 172.21.15.37:0/4186307318 conn(0x7f75d8076210 msgr2=0x7f75d8107860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 -- 172.21.15.37:0/4186307318 shutdown_connections 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.844+0000 7f75dfe00700 1 -- 172.21.15.37:0/4186307318 wait complete. 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.845+0000 7f75dfe00700 1 Processor -- start 2024-04-26T02:08:43.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.845+0000 7f75dfe00700 1 -- start start 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 0x7f75d819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 0x7f75d819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dfe00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d810e3b0 0x7f75d81a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dfe00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8119290 con 0x7f75d81092d0 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dfe00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8119110 con 0x7f75d8109c80 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dfe00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8119410 con 0x7f75d810e3b0 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dd400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 0x7f75d819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 0x7f75d819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dca00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 0x7f75d819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:56320/0 (socket says 172.21.15.37:56320) 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dca00700 1 -- 172.21.15.37:0/3236407102 learned_addr learned my addr 172.21.15.37:0/3236407102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:43.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.846+0000 7f75dde00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d810e3b0 0x7f75d81a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dca00700 1 -- 172.21.15.37:0/3236407102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d810e3b0 msgr2=0x7f75d81a4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dca00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d810e3b0 0x7f75d81a4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dca00700 1 -- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 msgr2=0x7f75d819f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dca00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 0x7f75d819f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dca00700 1 -- 172.21.15.37:0/3236407102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81a4760 con 0x7f75d8109c80 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dca00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 0x7f75d819fce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f75c400e970 tx=0x7f75c400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004a30 con 0x7f75d8109c80 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c400cc70 con 0x7f75d8109c80 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d81a4a50 con 0x7f75d8109c80 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4005580 con 0x7f75d8109c80 2024-04-26T02:08:43.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.847+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81ac2d0 con 0x7f75d8109c80 2024-04-26T02:08:43.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.848+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8069730 con 0x7f75d8109c80 2024-04-26T02:08:43.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.850+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f75c400c7c0 con 0x7f75d8109c80 2024-04-26T02:08:43.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.852+0000 7f75d6000700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f75c00a3e30 0x7f75c00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:43.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.852+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f75c40d0990 con 0x7f75d8109c80 2024-04-26T02:08:43.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.852+0000 7f75dd400700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f75c00a3e30 0x7f75c00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:43.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.853+0000 7f75dd400700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f75c00a3e30 0x7f75c00a62f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc005fb0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:43.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:43.856+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75c409cfd0 con 0x7f75d8109c80 2024-04-26T02:08:44.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.108+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f75d802df80 con 0x7f75d8109c80 2024-04-26T02:08:44.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.108+0000 7f75d6000700 1 -- 172.21.15.37:0/3236407102 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v100) ==== 81+0+938 (secure 0 0 0) 0x7f75c400ce20 con 0x7f75d8109c80 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/4056686696 2024-04-27T01:57:56.610509+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/793487795 2024-04-27T01:57:56.610509+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/29145428 2024-04-27T01:57:56.610509+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/457714458 2024-04-27T01:57:56.610509+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/691602839 2024-04-27T01:57:56.610509+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2267935573 2024-04-27T01:45:42.363226+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/691602839 2024-04-27T01:57:56.610509+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1381224435 2024-04-27T01:45:42.363226+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2267935573 2024-04-27T01:45:42.363226+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3897485386 2024-04-27T01:45:42.363226+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/1210493699 2024-04-27T01:45:06.740013+0000 2024-04-26T02:08:44.109 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/1210493699 2024-04-27T01:45:06.740013+0000 2024-04-26T02:08:44.110 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2037138951 2024-04-27T01:45:06.740013+0000 2024-04-26T02:08:44.110 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/19474513 2024-04-27T01:45:06.740013+0000 2024-04-26T02:08:44.110 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3832279297 2024-04-27T01:45:42.363226+0000 2024-04-26T02:08:44.110 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2777822460 2024-04-27T01:45:06.740013+0000 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f75c00a3e30 msgr2=0x7f75c00a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f75c00a3e30 0x7f75c00a62f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75cc000c00 tx=0x7f75cc005fb0 comp rx=0 tx=0).stop 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 msgr2=0x7f75d819fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 0x7f75d819fce0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f75c400e970 tx=0x7f75c400ec80 comp rx=0 tx=0).stop 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 shutdown_connections 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75d81092d0 0x7f75d819f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f75c00a3e30 0x7f75c00a62f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f75d810e3b0 0x7f75d81a4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 --2- 172.21.15.37:0/3236407102 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f75d8109c80 0x7f75d819fce0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.111+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 >> 172.21.15.37:0/3236407102 conn(0x7f75d8076210 msgr2=0x7f75d8077e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.112+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 shutdown_connections 2024-04-26T02:08:44.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:44.112+0000 7f75dfe00700 1 -- 172.21.15.37:0/3236407102 wait complete. 2024-04-26T02:08:44.114 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-04-26T02:08:44.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:44 smithi071 ceph-mon[32578]: pgmap v334: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:44.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:44 smithi037 ceph-mon[27906]: pgmap v334: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:44.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:44 smithi045 ceph-mon[32520]: pgmap v334: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-04-26T02:08:44.930 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T02:08:44.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:08:44.931 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T02:08:45.065 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T02:08:45.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:08:45.065 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T02:08:45.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:08:45.147 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-04-26T02:08:45.186 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T02:08:45.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:08:45.187 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T02:08:45.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:45 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3236407102' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:08:45.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T02:08:45.277 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-04-26T02:08:45.392 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T02:08:45.402 INFO:tasks.cephadm:Teardown begin 2024-04-26T02:08:45.403 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T02:08:45.403 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:08:45.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:45 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3236407102' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:08:45.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:45 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3236407102' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T02:08:45.846 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:46.284 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:46 smithi037 ceph-mon[27906]: pgmap v335: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:46.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:46 smithi071 ceph-mon[32578]: pgmap v335: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:46.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:46 smithi045 ceph-mon[32520]: pgmap v335: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:48.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.272+0000 7faea3e00700 1 -- 172.21.15.37:0/3435873160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06a750 msgr2=0x7fae9c06abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:48.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.272+0000 7faea3e00700 1 --2- 172.21.15.37:0/3435873160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06a750 0x7fae9c06abb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fae88009990 tx=0x7fae88009ca0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 -- 172.21.15.37:0/3435873160 shutdown_connections 2024-04-26T02:08:48.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 --2- 172.21.15.37:0/3435873160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c108a30 0x7fae9c108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 --2- 172.21.15.37:0/3435873160 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9c06b0f0 0x7fae9c10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 --2- 172.21.15.37:0/3435873160 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06a750 0x7fae9c06abb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 -- 172.21.15.37:0/3435873160 >> 172.21.15.37:0/3435873160 conn(0x7fae9c0788f0 msgr2=0x7fae9c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:48.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 -- 172.21.15.37:0/3435873160 shutdown_connections 2024-04-26T02:08:48.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.273+0000 7faea3e00700 1 -- 172.21.15.37:0/3435873160 wait complete. 2024-04-26T02:08:48.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.274+0000 7faea3e00700 1 Processor -- start 2024-04-26T02:08:48.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.274+0000 7faea3e00700 1 -- start start 2024-04-26T02:08:48.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea3e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c06a750 0x7fae9c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:48.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea3e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06b0f0 0x7fae9c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:48.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea3e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9c108a30 0x7fae9c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:48.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea3e00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c112bd0 con 0x7fae9c06a750 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea3e00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c112a50 con 0x7fae9c06b0f0 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea3e00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fae9c112d50 con 0x7fae9c108a30 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c06a750 0x7fae9c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea1400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c06a750 0x7fae9c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38706/0 (socket says 172.21.15.37:38706) 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea1400700 1 -- 172.21.15.37:0/2277719224 learned_addr learned my addr 172.21.15.37:0/2277719224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea0a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06b0f0 0x7fae9c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:48.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.275+0000 7faea1e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9c108a30 0x7fae9c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.276+0000 7faea0a00700 1 -- 172.21.15.37:0/2277719224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9c108a30 msgr2=0x7fae9c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.276+0000 7faea0a00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9c108a30 0x7fae9c19da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.276+0000 7faea0a00700 1 -- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c06a750 msgr2=0x7fae9c199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.276+0000 7faea0a00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c06a750 0x7fae9c199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.276+0000 7faea0a00700 1 -- 172.21.15.37:0/2277719224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9c19e1a0 con 0x7fae9c06b0f0 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.276+0000 7faea0a00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06b0f0 0x7fae9c1996d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae880059d0 tx=0x7fae880047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:48.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.277+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88027070 con 0x7fae9c06b0f0 2024-04-26T02:08:48.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.277+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae88022470 con 0x7fae9c06b0f0 2024-04-26T02:08:48.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.277+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae88017650 con 0x7fae9c06b0f0 2024-04-26T02:08:48.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.277+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae9c19e430 con 0x7fae9c06b0f0 2024-04-26T02:08:48.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.277+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae9c19e890 con 0x7fae9c06b0f0 2024-04-26T02:08:48.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.279+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae9c069730 con 0x7fae9c06b0f0 2024-04-26T02:08:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.280+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fae8802a070 con 0x7fae9c06b0f0 2024-04-26T02:08:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.282+0000 7fae9a000700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fae840a3db0 0x7fae840a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.282+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fae880d1410 con 0x7fae9c06b0f0 2024-04-26T02:08:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.282+0000 7faea1400700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fae840a3db0 0x7fae840a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:48.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.283+0000 7faea1400700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fae840a3db0 0x7fae840a6270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ca0 tx=0x7fae90009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:48.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.286+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fae9c19e890 con 0x7fae9c06b0f0 2024-04-26T02:08:48.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:48 smithi071 ceph-mon[32578]: pgmap v336: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:48.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:48 smithi037 ceph-mon[27906]: pgmap v336: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:48.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:48 smithi045 ceph-mon[32520]: pgmap v336: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 852 B/s rd, 1 op/s 2024-04-26T02:08:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.568+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae9c02df80 con 0x7fae9c06b0f0 2024-04-26T02:08:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.569+0000 7fae9a000700 1 -- 172.21.15.37:0/2277719224 <== mon.2 v2:172.21.15.45:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fae88013070 con 0x7fae9c06b0f0 2024-04-26T02:08:48.569 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:08:48.569 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fae840a3db0 msgr2=0x7fae840a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fae840a3db0 0x7fae840a6270 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fae90009ca0 tx=0x7fae90009260 comp rx=0 tx=0).stop 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06b0f0 msgr2=0x7fae9c1996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06b0f0 0x7fae9c1996d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fae880059d0 tx=0x7fae880047b0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 shutdown_connections 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae9c06a750 0x7fae9c199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fae840a3db0 0x7fae840a6270 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fae9c108a30 0x7fae9c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 --2- 172.21.15.37:0/2277719224 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fae9c06b0f0 0x7fae9c1996d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.572+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 >> 172.21.15.37:0/2277719224 conn(0x7fae9c0788f0 msgr2=0x7fae9c0ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.573+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 shutdown_connections 2024-04-26T02:08:48.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:48.573+0000 7faea3e00700 1 -- 172.21.15.37:0/2277719224 wait complete. 2024-04-26T02:08:48.573 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:08:49.190 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:49 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/2277719224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:08:49.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:49 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/2277719224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:08:49.450 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T02:08:49.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:49 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/2277719224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:08:49.891 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:50.391 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:50 smithi037 ceph-mon[27906]: pgmap v337: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:50.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:50 smithi071 ceph-mon[32578]: pgmap v337: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:50.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:50 smithi045 ceph-mon[32520]: pgmap v337: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:52.162 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:52 smithi037 ceph-mon[27906]: pgmap v338: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:52.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 -- 172.21.15.37:0/4136769747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 msgr2=0x7fc3b810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:52.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 --2- 172.21.15.37:0/4136769747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b810f900 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc3b000b0a0 tx=0x7fc3b000b3b0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 -- 172.21.15.37:0/4136769747 shutdown_connections 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 --2- 172.21.15.37:0/4136769747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8108a30 0x7fc3b8108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 --2- 172.21.15.37:0/4136769747 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b810f900 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 --2- 172.21.15.37:0/4136769747 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 0x7fc3b806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 -- 172.21.15.37:0/4136769747 >> 172.21.15.37:0/4136769747 conn(0x7fc3b80788f0 msgr2=0x7fc3b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 -- 172.21.15.37:0/4136769747 shutdown_connections 2024-04-26T02:08:52.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.190+0000 7fc3bd200700 1 -- 172.21.15.37:0/4136769747 wait complete. 2024-04-26T02:08:52.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.191+0000 7fc3bd200700 1 Processor -- start 2024-04-26T02:08:52.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 -- start start 2024-04-26T02:08:52.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 0x7fc3b819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:52.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8108a30 0x7fc3b819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8112bd0 con 0x7fc3b8108a30 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8112a50 con 0x7fc3b806a750 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3bd200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fc3b8112d50 con 0x7fc3b806b0f0 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3b5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 0x7fc3b819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3b5e00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 0x7fc3b819b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:50920/0 (socket says 172.21.15.37:50920) 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3b5e00700 1 -- 172.21.15.37:0/990714148 learned_addr learned my addr 172.21.15.37:0/990714148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3ad400700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:52.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.192+0000 7fc3b6800700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8108a30 0x7fc3b819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.193+0000 7fc3ad400700 1 -- 172.21.15.37:0/990714148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 msgr2=0x7fc3b819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.193+0000 7fc3ad400700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 0x7fc3b819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.193+0000 7fc3ad400700 1 -- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8108a30 msgr2=0x7fc3b819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.193+0000 7fc3ad400700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8108a30 0x7fc3b819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.193+0000 7fc3ad400700 1 -- 172.21.15.37:0/990714148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3b81a0330 con 0x7fc3b806b0f0 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.193+0000 7fc3ad400700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b819b860 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000e9f0 tx=0x7fc3a000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.194+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a000cb70 con 0x7fc3b806b0f0 2024-04-26T02:08:52.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.194+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3b81a0620 con 0x7fc3b806b0f0 2024-04-26T02:08:52.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.194+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3a000ee90 con 0x7fc3b806b0f0 2024-04-26T02:08:52.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.194+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a0003d40 con 0x7fc3b806b0f0 2024-04-26T02:08:52.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.194+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3b81a7ea0 con 0x7fc3b806b0f0 2024-04-26T02:08:52.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.195+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3b8069730 con 0x7fc3b806b0f0 2024-04-26T02:08:52.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.197+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3a0003ee0 con 0x7fc3b806b0f0 2024-04-26T02:08:52.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.199+0000 7fc3af400700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc3980a3db0 0x7fc3980a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:52.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.199+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7fc3a00d0920 con 0x7fc3b806b0f0 2024-04-26T02:08:52.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.199+0000 7fc3b5e00700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc3980a3db0 0x7fc3980a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:52.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.200+0000 7fc3b5e00700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc3980a3db0 0x7fc3980a6270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009ca0 tx=0x7fc3a8009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:52.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.203+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3a009cf60 con 0x7fc3b806b0f0 2024-04-26T02:08:52.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:52 smithi071 ceph-mon[32578]: pgmap v338: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:52.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.492+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3b819c680 con 0x7fc3b806b0f0 2024-04-26T02:08:52.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.493+0000 7fc3af400700 1 -- 172.21.15.37:0/990714148 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+11781 (secure 0 0 0) 0x7fc3a009c860 con 0x7fc3b806b0f0 2024-04-26T02:08:52.493 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:08:52.493 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14403,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2822754454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2822754454},{"type":"v1","addr":"172.21.15.37:6841","nonce":2822754454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6833/2132222354","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6832","nonce":2132222354},{"type":"v1","addr":"172.21.15.45:6833","nonce":2132222354}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/3616070379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":3616070379},{"type":"v1","addr":"172.21.15.37:6835","nonce":3616070379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/200842518","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":200842518},{"type":"v1","addr":"172.21.15.37:6837","nonce":200842518}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6833/10421136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6832","nonce":10421136},{"type":"v1","addr":"172.21.15.71:6833","nonce":10421136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24458,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6835/3667220956","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6834","nonce":3667220956},{"type":"v1","addr":"172.21.15.45:6835","nonce":3667220956}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.45:6837/1744923688","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6836","nonce":1744923688},{"type":"v1","addr":"172.21.15.45:6837","nonce":1744923688}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24485,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/1856429738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":1856429738},{"type":"v1","addr":"172.21.15.37:6839","nonce":1856429738}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24490,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6835/1784845440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6834","nonce":1784845440},{"type":"v1","addr":"172.21.15.71:6835","nonce":1784845440}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24502,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.71:6837/306008379","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6836","nonce":306008379},{"type":"v1","addr":"172.21.15.71:6837","nonce":306008379}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T01:54:12.162501+0000","modified":"2024-04-26T01:54:33.402956+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24514},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24509":{"gid":24509,"name":"b","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.45:6839/60713297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6838","nonce":60713297},{"type":"v1","addr":"172.21.15.45:6839","nonce":60713297}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24514":{"gid":24514,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":11,"addr":"172.21.15.71:6839/1148187051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6838","nonce":1148187051},{"type":"v1","addr":"172.21.15.71:6839","nonce":1148187051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24514,"qdb_cluster":[24514]},"id":1}]} 2024-04-26T02:08:52.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc3980a3db0 msgr2=0x7fc3980a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc3980a3db0 0x7fc3980a6270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009ca0 tx=0x7fc3a8009260 comp rx=0 tx=0).stop 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 msgr2=0x7fc3b819b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b819b860 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc3a000e9f0 tx=0x7fc3a000ed00 comp rx=0 tx=0).stop 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 shutdown_connections 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc3b8108a30 0x7fc3b819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fc3980a3db0 0x7fc3980a6270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fc3b806a750 0x7fc3b819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 --2- 172.21.15.37:0/990714148 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fc3b806b0f0 0x7fc3b819b860 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:52.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 >> 172.21.15.37:0/990714148 conn(0x7fc3b80788f0 msgr2=0x7fc3b810e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:52.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 shutdown_connections 2024-04-26T02:08:52.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:52.497+0000 7fc3bd200700 1 -- 172.21.15.37:0/990714148 wait complete. 2024-04-26T02:08:52.498 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 19 2024-04-26T02:08:52.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:52 smithi045 ceph-mon[32520]: pgmap v338: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:53.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:53 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/990714148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:08:53.336 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T02:08:53.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:53 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/990714148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:08:53.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:53 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/990714148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T02:08:53.815 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:54.354 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:54 smithi037 ceph-mon[27906]: pgmap v339: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:54.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:54 smithi071 ceph-mon[32578]: pgmap v339: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:54.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:54 smithi045 ceph-mon[32520]: pgmap v339: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 -- 172.21.15.37:0/3961794348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c108a50 msgr2=0x7f4f0c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 --2- 172.21.15.37:0/3961794348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c108a50 0x7f4f0c108e30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009990 tx=0x7f4efc009ca0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 -- 172.21.15.37:0/3961794348 shutdown_connections 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 --2- 172.21.15.37:0/3961794348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f0c103450 0x7f4f0c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 --2- 172.21.15.37:0/3961794348 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c108a50 0x7f4f0c108e30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 --2- 172.21.15.37:0/3961794348 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c102a90 0x7f4f0c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.045+0000 7f4f11200700 1 -- 172.21.15.37:0/3961794348 >> 172.21.15.37:0/3961794348 conn(0x7f4f0c079ba0 msgr2=0x7f4f0c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.046+0000 7f4f11200700 1 -- 172.21.15.37:0/3961794348 shutdown_connections 2024-04-26T02:08:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.046+0000 7f4f11200700 1 -- 172.21.15.37:0/3961794348 wait complete. 2024-04-26T02:08:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 Processor -- start 2024-04-26T02:08:56.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 -- start start 2024-04-26T02:08:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f0c102a90 0x7f4f0c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c103450 0x7f4f0c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 0x7f4f0c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c112c60 con 0x7f4f0c102a90 2024-04-26T02:08:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c112ae0 con 0x7f4f0c103450 2024-04-26T02:08:56.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.047+0000 7f4f11200700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c112de0 con 0x7f4f0c108a50 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 0x7f4f0c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 0x7f4f0c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:37722/0 (socket says 172.21.15.37:37722) 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 -- 172.21.15.37:0/3128715659 learned_addr learned my addr 172.21.15.37:0/3128715659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f09e00700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f0c102a90 0x7f4f0c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f09400700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c103450 0x7f4f0c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 -- 172.21.15.37:0/3128715659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c103450 msgr2=0x7f4f0c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c103450 0x7f4f0c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 -- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f0c102a90 msgr2=0x7f4f0c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f0c102a90 0x7f4f0c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f0a800700 1 -- 172.21.15.37:0/3128715659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c1a0440 con 0x7f4f0c108a50 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.048+0000 7f4f09400700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c103450 0x7f4f0c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.049+0000 7f4f0a800700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 0x7f4f0c19fd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f04007ee0 tx=0x7f4f0400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.049+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04004bb0 con 0x7f4f0c108a50 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.049+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c1a0730 con 0x7f4f0c108a50 2024-04-26T02:08:56.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.049+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f04004d50 con 0x7f4f0c108a50 2024-04-26T02:08:56.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.049+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04005920 con 0x7f4f0c108a50 2024-04-26T02:08:56.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.049+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c1a7fb0 con 0x7f4f0c108a50 2024-04-26T02:08:56.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.050+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f0c069730 con 0x7f4f0c108a50 2024-04-26T02:08:56.051 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.052+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f0401a020 con 0x7f4f0c108a50 2024-04-26T02:08:56.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.054+0000 7f4f02a00700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4ef00a3db0 0x7f4ef00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:56.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.054+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f4f040d0160 con 0x7f4f0c108a50 2024-04-26T02:08:56.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.054+0000 7f4f09e00700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4ef00a3db0 0x7f4ef00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:56.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.055+0000 7f4f09e00700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4ef00a3db0 0x7f4ef00a6270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00b5c0 tx=0x7f4efc005fd0 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:56.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.056+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f04010070 con 0x7f4f0c108a50 2024-04-26T02:08:56.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:56 smithi037 ceph-mon[27906]: pgmap v340: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:56.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.290+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f0c1a8180 con 0x7f4f0c108a50 2024-04-26T02:08:56.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.292+0000 7f4f02a00700 1 -- 172.21.15.37:0/3128715659 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+22962 (secure 0 0 0) 0x7f4f0409c7a0 con 0x7f4f0c108a50 2024-04-26T02:08:56.292 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-04-26T02:08:56.292 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":100,"fsid":"5d371fa8-036e-11ef-bc93-c7b262605968","created":"2024-04-26T01:44:24.368582+0000","modified":"2024-04-26T02:08:09.864791+0000","last_up_change":"2024-04-26T01:52:48.799333+0000","last_in_change":"2024-04-26T01:52:27.648747+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T01:48:32.516892+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T01:54:02.892469+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T01:54:06.974700+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d2501dac-396b-46d1-a9ff-b3b157f46fb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6803","nonce":2603696107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6805","nonce":2603696107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6809","nonce":2603696107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2603696107},{"type":"v1","addr":"172.21.15.37:6807","nonce":2603696107}]},"public_addr":"172.21.15.37:6803/2603696107","cluster_addr":"172.21.15.37:6805/2603696107","heartbeat_back_addr":"172.21.15.37:6809/2603696107","heartbeat_front_addr":"172.21.15.37:6807/2603696107","state":["exists","up"]},{"osd":1,"uuid":"f91f576c-313e-4a6a-9f40-715003e44614","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6800","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6801","nonce":3445601743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6802","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6803","nonce":3445601743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6806","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6807","nonce":3445601743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6804","nonce":3445601743},{"type":"v1","addr":"172.21.15.45:6805","nonce":3445601743}]},"public_addr":"172.21.15.45:6801/3445601743","cluster_addr":"172.21.15.45:6803/3445601743","heartbeat_back_addr":"172.21.15.45:6807/3445601743","heartbeat_front_addr":"172.21.15.45:6805/3445601743","state":["exists","up"]},{"osd":2,"uuid":"a90c81da-6303-4de4-8556-bd4ba8819380","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6800","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6801","nonce":2117420904}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6802","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6803","nonce":2117420904}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6806","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6807","nonce":2117420904}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6804","nonce":2117420904},{"type":"v1","addr":"172.21.15.71:6805","nonce":2117420904}]},"public_addr":"172.21.15.71:6801/2117420904","cluster_addr":"172.21.15.71:6803/2117420904","heartbeat_back_addr":"172.21.15.71:6807/2117420904","heartbeat_front_addr":"172.21.15.71:6805/2117420904","state":["exists","up"]},{"osd":3,"uuid":"950c30ee-de7f-461e-89b8-5218e34caf03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6811","nonce":1364499225}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6813","nonce":1364499225}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6817","nonce":1364499225}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1364499225},{"type":"v1","addr":"172.21.15.37:6815","nonce":1364499225}]},"public_addr":"172.21.15.37:6811/1364499225","cluster_addr":"172.21.15.37:6813/1364499225","heartbeat_back_addr":"172.21.15.37:6817/1364499225","heartbeat_front_addr":"172.21.15.37:6815/1364499225","state":["exists","up"]},{"osd":4,"uuid":"b70dc87f-17e7-44e4-b5d1-80abdcc91aed","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6808","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6809","nonce":2655399614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6810","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6811","nonce":2655399614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6814","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6815","nonce":2655399614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6812","nonce":2655399614},{"type":"v1","addr":"172.21.15.45:6813","nonce":2655399614}]},"public_addr":"172.21.15.45:6809/2655399614","cluster_addr":"172.21.15.45:6811/2655399614","heartbeat_back_addr":"172.21.15.45:6815/2655399614","heartbeat_front_addr":"172.21.15.45:6813/2655399614","state":["exists","up"]},{"osd":5,"uuid":"bbe95809-3e82-4167-8b84-fed9410871e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6808","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6809","nonce":1060444370}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6810","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6811","nonce":1060444370}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6814","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6815","nonce":1060444370}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6812","nonce":1060444370},{"type":"v1","addr":"172.21.15.71:6813","nonce":1060444370}]},"public_addr":"172.21.15.71:6809/1060444370","cluster_addr":"172.21.15.71:6811/1060444370","heartbeat_back_addr":"172.21.15.71:6815/1060444370","heartbeat_front_addr":"172.21.15.71:6813/1060444370","state":["exists","up"]},{"osd":6,"uuid":"1af9e89d-771d-4ce4-8e7b-693cf99fe580","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6819","nonce":2087418626}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6821","nonce":2087418626}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6825","nonce":2087418626}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":2087418626},{"type":"v1","addr":"172.21.15.37:6823","nonce":2087418626}]},"public_addr":"172.21.15.37:6819/2087418626","cluster_addr":"172.21.15.37:6821/2087418626","heartbeat_back_addr":"172.21.15.37:6825/2087418626","heartbeat_front_addr":"172.21.15.37:6823/2087418626","state":["exists","up"]},{"osd":7,"uuid":"3389b007-2053-4508-a620-d820a3a6fecf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6816","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6817","nonce":1922916284}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6818","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6819","nonce":1922916284}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6822","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6823","nonce":1922916284}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6820","nonce":1922916284},{"type":"v1","addr":"172.21.15.45:6821","nonce":1922916284}]},"public_addr":"172.21.15.45:6817/1922916284","cluster_addr":"172.21.15.45:6819/1922916284","heartbeat_back_addr":"172.21.15.45:6823/1922916284","heartbeat_front_addr":"172.21.15.45:6821/1922916284","state":["exists","up"]},{"osd":8,"uuid":"ad43bcc8-3414-401c-bbff-6ff4ed6a3149","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6816","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6817","nonce":1870194976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6818","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6819","nonce":1870194976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6822","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6823","nonce":1870194976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6820","nonce":1870194976},{"type":"v1","addr":"172.21.15.71:6821","nonce":1870194976}]},"public_addr":"172.21.15.71:6817/1870194976","cluster_addr":"172.21.15.71:6819/1870194976","heartbeat_back_addr":"172.21.15.71:6823/1870194976","heartbeat_front_addr":"172.21.15.71:6821/1870194976","state":["exists","up"]},{"osd":9,"uuid":"22321723-8f84-485f-9ecc-fe50bf7c0233","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6827","nonce":759844369}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6829","nonce":759844369}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6833","nonce":759844369}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":759844369},{"type":"v1","addr":"172.21.15.37:6831","nonce":759844369}]},"public_addr":"172.21.15.37:6827/759844369","cluster_addr":"172.21.15.37:6829/759844369","heartbeat_back_addr":"172.21.15.37:6833/759844369","heartbeat_front_addr":"172.21.15.37:6831/759844369","state":["exists","up"]},{"osd":10,"uuid":"f2b2d29e-56a1-427e-a87f-911da6c8adf4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6824","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6825","nonce":685235044}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6826","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6827","nonce":685235044}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6830","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6831","nonce":685235044}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.45:6828","nonce":685235044},{"type":"v1","addr":"172.21.15.45:6829","nonce":685235044}]},"public_addr":"172.21.15.45:6825/685235044","cluster_addr":"172.21.15.45:6827/685235044","heartbeat_back_addr":"172.21.15.45:6831/685235044","heartbeat_front_addr":"172.21.15.45:6829/685235044","state":["exists","up"]},{"osd":11,"uuid":"16fcf494-d10b-4505-a211-636f44fcf2f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6824","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6825","nonce":584658428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6826","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6827","nonce":584658428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6830","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6831","nonce":584658428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.71:6828","nonce":584658428},{"type":"v1","addr":"172.21.15.71:6829","nonce":584658428}]},"public_addr":"172.21.15.71:6825/584658428","cluster_addr":"172.21.15.71:6827/584658428","heartbeat_back_addr":"172.21.15.71:6831/584658428","heartbeat_front_addr":"172.21.15.71:6829/584658428","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:47:39.725664+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:05.106997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:48:29.108344+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:03.124010+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:28.797903+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:49:53.033530+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:28.248249+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:50:54.281242+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:19.032476+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:51:54.126860+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:20.482744+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T01:52:45.815752+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.6","mappings":[{"from":4,"to":7}]},{"pgid":"3.b","mappings":[{"from":3,"to":9}]},{"pgid":"3.20","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":4,"to":8}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/4056686696":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/793487795":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/29145428":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/457714458":"2024-04-27T01:57:56.610509+0000","172.21.15.37:6800/691602839":"2024-04-27T01:57:56.610509+0000","172.21.15.37:6801/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/691602839":"2024-04-27T01:57:56.610509+0000","172.21.15.37:0/1381224435":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6800/2267935573":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/3897485386":"2024-04-27T01:45:42.363226+0000","172.21.15.37:6801/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:6800/1210493699":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/2037138951":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/19474513":"2024-04-27T01:45:06.740013+0000","172.21.15.37:0/3832279297":"2024-04-27T01:45:42.363226+0000","172.21.15.37:0/2777822460":"2024-04-27T01:45:06.740013+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4ef00a3db0 msgr2=0x7f4ef00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4ef00a3db0 0x7f4ef00a6270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4efc00b5c0 tx=0x7f4efc005fd0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 msgr2=0x7f4f0c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 0x7f4f0c19fd00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f04007ee0 tx=0x7f4f0400d3b0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 shutdown_connections 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f0c102a90 0x7f4f0c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f4ef00a3db0 0x7f4ef00a6270 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f4f0c108a50 0x7f4f0c19fd00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 --2- 172.21.15.37:0/3128715659 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f4f0c103450 0x7f4f0c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.295+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 >> 172.21.15.37:0/3128715659 conn(0x7f4f0c079ba0 msgr2=0x7f4f0c10df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.296+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 shutdown_connections 2024-04-26T02:08:56.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:56.296+0000 7f4f11200700 1 -- 172.21.15.37:0/3128715659 wait complete. 2024-04-26T02:08:56.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:56 smithi071 ceph-mon[32578]: pgmap v340: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:56.504 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:56 smithi045 ceph-mon[32520]: pgmap v340: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:57.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-26T02:08:57.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:57 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/3128715659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:08:57.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:57 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/3128715659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:08:57.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:57 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/3128715659' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T02:08:57.600 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:08:58.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:08:58 smithi071 ceph-mon[32578]: pgmap v341: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:58.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:08:58 smithi037 ceph-mon[27906]: pgmap v341: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:58.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:08:58 smithi045 ceph-mon[32520]: pgmap v341: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:08:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 -- 172.21.15.37:0/218689029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8418075590 msgr2=0x7f841810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 --2- 172.21.15.37:0/218689029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8418075590 0x7f841810ef00 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8404009990 tx=0x7f8404009ca0 comp rx=0 tx=0).stop 2024-04-26T02:08:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 -- 172.21.15.37:0/218689029 shutdown_connections 2024-04-26T02:08:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 --2- 172.21.15.37:0/218689029 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f841810f440 0x7f8418111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:59.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 --2- 172.21.15.37:0/218689029 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418074c70 0x7f8418075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 --2- 172.21.15.37:0/218689029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8418075590 0x7f841810ef00 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 -- 172.21.15.37:0/218689029 >> 172.21.15.37:0/218689029 conn(0x7f8418079ba0 msgr2=0x7f8418079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:08:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 -- 172.21.15.37:0/218689029 shutdown_connections 2024-04-26T02:08:59.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.841+0000 7f841ec00700 1 -- 172.21.15.37:0/218689029 wait complete. 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.842+0000 7f841ec00700 1 Processor -- start 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 -- start start 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8418074c70 0x7f841819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418075590 0x7f841819b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 0x7f841819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8418114e70 con 0x7f841810f440 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f8418114cf0 con 0x7f8418075590 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841ec00700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f8418114ff0 con 0x7f8418074c70 2024-04-26T02:08:59.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 0x7f841819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841cc00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 0x7f841819fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59358/0 (socket says 172.21.15.37:59358) 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.843+0000 7f841cc00700 1 -- 172.21.15.37:0/1421657246 learned_addr learned my addr 172.21.15.37:0/1421657246 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f8417e00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8418074c70 0x7f841819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f8417400700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418075590 0x7f841819b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f841cc00700 1 -- 172.21.15.37:0/1421657246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8418074c70 msgr2=0x7f841819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f841cc00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8418074c70 0x7f841819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f841cc00700 1 -- 172.21.15.37:0/1421657246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418075590 msgr2=0x7f841819b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f841cc00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418075590 0x7f841819b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f841cc00700 1 -- 172.21.15.37:0/1421657246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84181a0330 con 0x7f841810f440 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f8417e00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8418074c70 0x7f841819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:08:59.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f8417400700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418075590 0x7f841819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T02:08:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.844+0000 7f841cc00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 0x7f841819fbf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f841000ef90 tx=0x7f841000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.845+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100092e0 con 0x7f841810f440 2024-04-26T02:08:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.845+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8410004bb0 con 0x7f841810f440 2024-04-26T02:08:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.845+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84181a05c0 con 0x7f841810f440 2024-04-26T02:08:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.845+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410004280 con 0x7f841810f440 2024-04-26T02:08:59.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.845+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84181a7ea0 con 0x7f841810f440 2024-04-26T02:08:59.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.846+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8418069730 con 0x7f841810f440 2024-04-26T02:08:59.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.849+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f841001b030 con 0x7f841810f440 2024-04-26T02:08:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.851+0000 7f8414c00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f83fc0a3e30 0x7f83fc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:08:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.851+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8513+0+0 (secure 0 0 0) 0x7f84100d0d70 con 0x7f841810f440 2024-04-26T02:08:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.851+0000 7f8417e00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f83fc0a3e30 0x7f83fc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:08:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.852+0000 7f8417e00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f83fc0a3e30 0x7f83fc0a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8408009680 tx=0x7f8408009260 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:08:59.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:08:59.854+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8410022020 con 0x7f841810f440 2024-04-26T02:09:00.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.138+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f841802df80 con 0x7f841810f440 2024-04-26T02:09:00.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.143+0000 7f8414c00700 1 -- 172.21.15.37:0/1421657246 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v20) ==== 161+0+0 (secure 0 0 0) 0x7f8410010040 con 0x7f841810f440 2024-04-26T02:09:00.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f83fc0a3e30 msgr2=0x7f83fc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:00.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f83fc0a3e30 0x7f83fc0a62f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8408009680 tx=0x7f8408009260 comp rx=0 tx=0).stop 2024-04-26T02:09:00.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 msgr2=0x7f841819fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 0x7f841819fbf0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f841000ef90 tx=0x7f841000c3a0 comp rx=0 tx=0).stop 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 shutdown_connections 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f841810f440 0x7f841819fbf0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.148+0000 7f841ec00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f83fc0a3e30 0x7f83fc0a62f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.149+0000 7f841ec00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f8418074c70 0x7f841819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.149+0000 7f841ec00700 1 --2- 172.21.15.37:0/1421657246 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f8418075590 0x7f841819b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:00.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.149+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 >> 172.21.15.37:0/1421657246 conn(0x7f8418079ba0 msgr2=0x7f84180ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.149+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 shutdown_connections 2024-04-26T02:09:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:00.149+0000 7f841ec00700 1 -- 172.21.15.37:0/1421657246 wait complete. 2024-04-26T02:09:00.149 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T02:09:00.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:00 smithi071 ceph-mon[32578]: pgmap v342: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:09:00.403 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:09:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: ignoring --setuser ceph since I am not root 2024-04-26T02:09:00.403 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:09:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: ignoring --setgroup ceph since I am not root 2024-04-26T02:09:00.403 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:09:00 smithi071 ceph-mds[63319]: -- 172.21.15.71:0/3417959077 <== mon.1 v2:172.21.15.71:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560685d8ba40 con 0x560685d5ec00 2024-04-26T02:09:00.403 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:09:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: starting mds.i at 2024-04-26T02:09:00.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:00 smithi037 ceph-mon[27906]: pgmap v342: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:09:00.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a[27902]: 2024-04-26T02:09:00.140+0000 7f434a800700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:09:00.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:00 smithi045 ceph-mon[32520]: pgmap v342: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 255 B/s wr, 2 op/s 2024-04-26T02:09:00.505 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:09:00 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b[63673]: ignoring --setuser ceph since I am not root 2024-04-26T02:09:00.505 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:09:00 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b[63673]: ignoring --setgroup ceph since I am not root 2024-04-26T02:09:00.505 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:09:00 smithi045 ceph-mds[63677]: -- 172.21.15.45:0/2485705026 <== mon.2 v2:172.21.15.45:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e0d2457a40 con 0x55e0d242a800 2024-04-26T02:09:00.506 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:09:00 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b[63673]: starting mds.b at 2024-04-26T02:09:00.988 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T02:09:01.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:01 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1421657246' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:09:01.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:01 smithi037 ceph-mon[27906]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:09:01.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:01 smithi037 ceph-mon[27906]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:09:01.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:01 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1421657246' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:09:01.209 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:01 smithi037 ceph-mon[27906]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:09:01.210 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:01 smithi037 ceph-mon[27906]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T02:09:01.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:01 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1421657246' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:09:01.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:01 smithi071 ceph-mon[32578]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:09:01.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:01 smithi071 ceph-mon[32578]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:09:01.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:01 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1421657246' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:09:01.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:01 smithi071 ceph-mon[32578]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:09:01.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:01 smithi071 ceph-mon[32578]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T02:09:01.458 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:09:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:01 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1421657246' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T02:09:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:01 smithi045 ceph-mon[32520]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T02:09:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:01 smithi045 ceph-mon[32520]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T02:09:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:01 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1421657246' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T02:09:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:01 smithi045 ceph-mon[32520]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T02:09:01.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:01 smithi045 ceph-mon[32520]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-04-26T02:09:02.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:02 smithi071 ceph-mon[32578]: pgmap v344: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T02:09:02.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:02 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:09:02.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:02 smithi071 ceph-mon[32578]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:09:02.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:02 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.45:6838/1413108228,v1:172.21.15.45:6839/1413108228] up:boot 2024-04-26T02:09:02.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:02 smithi071 ceph-mon[32578]: mds.? [v2:172.21.15.71:6838/965828279,v1:172.21.15.71:6839/965828279] up:boot 2024-04-26T02:09:02.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:02 smithi071 ceph-mon[32578]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-04-26T02:09:02.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:02 smithi037 ceph-mon[27906]: pgmap v344: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T02:09:02.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:02 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:09:02.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:02 smithi037 ceph-mon[27906]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:09:02.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:02 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.45:6838/1413108228,v1:172.21.15.45:6839/1413108228] up:boot 2024-04-26T02:09:02.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:02 smithi037 ceph-mon[27906]: mds.? [v2:172.21.15.71:6838/965828279,v1:172.21.15.71:6839/965828279] up:boot 2024-04-26T02:09:02.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:02 smithi037 ceph-mon[27906]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-04-26T02:09:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:02 smithi045 ceph-mon[32520]: pgmap v344: 97 pgs: 97 active+clean; 2.7 MiB data, 610 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-04-26T02:09:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:02 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T02:09:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:02 smithi045 ceph-mon[32520]: from='mgr.14856 172.21.15.37:0/2754684395' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T02:09:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:02 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.45:6838/1413108228,v1:172.21.15.45:6839/1413108228] up:boot 2024-04-26T02:09:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:02 smithi045 ceph-mon[32520]: mds.? [v2:172.21.15.71:6838/965828279,v1:172.21.15.71:6839/965828279] up:boot 2024-04-26T02:09:02.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:02 smithi045 ceph-mon[32520]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-04-26T02:09:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 -- 172.21.15.37:0/2095544192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d4406a750 msgr2=0x7f6d4406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 --2- 172.21.15.37:0/2095544192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d4406a750 0x7f6d4406abb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f6d30009990 tx=0x7f6d30009ca0 comp rx=0 tx=0).stop 2024-04-26T02:09:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 -- 172.21.15.37:0/2095544192 shutdown_connections 2024-04-26T02:09:03.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 --2- 172.21.15.37:0/2095544192 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 0x7f6d4410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 --2- 172.21.15.37:0/2095544192 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d44108a30 0x7f6d44108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 --2- 172.21.15.37:0/2095544192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d4406a750 0x7f6d4406abb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 -- 172.21.15.37:0/2095544192 >> 172.21.15.37:0/2095544192 conn(0x7f6d440788f0 msgr2=0x7f6d44078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 -- 172.21.15.37:0/2095544192 shutdown_connections 2024-04-26T02:09:03.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.833+0000 7f6d4a000700 1 -- 172.21.15.37:0/2095544192 wait complete. 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.834+0000 7f6d4a000700 1 Processor -- start 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.834+0000 7f6d4a000700 1 -- start start 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d4a000700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d4406a750 0x7f6d4419b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d4a000700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 0x7f6d4419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d4a000700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d44108a30 0x7f6d4419fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d4a000700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44112bf0 con 0x7f6d44108a30 2024-04-26T02:09:03.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d4a000700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44112a70 con 0x7f6d4406b0f0 2024-04-26T02:09:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d4a000700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6d44112d70 con 0x7f6d4406a750 2024-04-26T02:09:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d42a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 0x7f6d4419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d42a00700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 0x7f6d4419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.45:3300/0 says I am v2:172.21.15.37:55976/0 (socket says 172.21.15.37:55976) 2024-04-26T02:09:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d4406a750 0x7f6d4419b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d42a00700 1 -- 172.21.15.37:0/1748131905 learned_addr learned my addr 172.21.15.37:0/1748131905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:09:03.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43e00700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d44108a30 0x7f6d4419fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43400700 1 -- 172.21.15.37:0/1748131905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 msgr2=0x7f6d4419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43400700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 0x7f6d4419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43400700 1 -- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d44108a30 msgr2=0x7f6d4419fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43400700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d44108a30 0x7f6d4419fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.835+0000 7f6d43400700 1 -- 172.21.15.37:0/1748131905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d441a0300 con 0x7f6d4406a750 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.836+0000 7f6d43400700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d4406a750 0x7f6d4419b340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400e9f0 tx=0x7f6d3400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.836+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d3400cb70 con 0x7f6d4406a750 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.836+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d441a05f0 con 0x7f6d4406a750 2024-04-26T02:09:03.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.836+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d3400ee90 con 0x7f6d4406a750 2024-04-26T02:09:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.836+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d34003d40 con 0x7f6d4406a750 2024-04-26T02:09:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.836+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d441a7e70 con 0x7f6d4406a750 2024-04-26T02:09:03.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.837+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d44069730 con 0x7f6d4406a750 2024-04-26T02:09:03.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.840+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6d34003ee0 con 0x7f6d4406a750 2024-04-26T02:09:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.841+0000 7f6d3be00700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6d280a3db0 0x7f6d280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.841+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8555+0+0 (secure 0 0 0) 0x7f6d340d08d0 con 0x7f6d4406a750 2024-04-26T02:09:03.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.841+0000 7f6d42a00700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6d280a3db0 0x7f6d280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:03.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.842+0000 7f6d42a00700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6d280a3db0 0x7f6d280a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6d30005cd0 tx=0x7f6d30005c60 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:09:03.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:03.843+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d3409cef0 con 0x7f6d4406a750 2024-04-26T02:09:04.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.124+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6d4402df80 con 0x7f6d4406a750 2024-04-26T02:09:04.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.158+0000 7f6d3be00700 1 -- 172.21.15.37:0/1748131905 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) ==== 104+0+0 (secure 0 0 0) 0x7f6d3409c7f0 con 0x7f6d4406a750 2024-04-26T02:09:04.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.162+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6d280a3db0 msgr2=0x7f6d280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.162+0000 7f6d4a000700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6d280a3db0 0x7f6d280a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6d30005cd0 tx=0x7f6d30005c60 comp rx=0 tx=0).stop 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.162+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d4406a750 msgr2=0x7f6d4419b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.162+0000 7f6d4a000700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d4406a750 0x7f6d4419b340 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6d3400e9f0 tx=0x7f6d3400ed00 comp rx=0 tx=0).stop 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 shutdown_connections 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d44108a30 0x7f6d4419fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6d280a3db0 0x7f6d280a6270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:04.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6d4406a750 0x7f6d4419b340 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:04.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 --2- 172.21.15.37:0/1748131905 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6d4406b0f0 0x7f6d4419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:04.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 >> 172.21.15.37:0/1748131905 conn(0x7f6d440788f0 msgr2=0x7f6d4410df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:04.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 shutdown_connections 2024-04-26T02:09:04.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:04.163+0000 7f6d4a000700 1 -- 172.21.15.37:0/1748131905 wait complete. 2024-04-26T02:09:04.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:04 smithi037 ceph-mon[27906]: pgmap v345: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:04.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:04 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1748131905' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:09:04.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:04 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:09:04.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:04 smithi045 ceph-mon[32520]: pgmap v345: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:04.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:04 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1748131905' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:09:04.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:04 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:09:04.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:04 smithi071 ceph-mon[32578]: pgmap v345: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:04.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:04 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1748131905' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:09:04.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:04 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T02:09:05.061 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T02:09:05.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:05 smithi037 ceph-mon[27906]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:09:05.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:05 smithi037 ceph-mon[27906]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:09:05.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:05 smithi037 ceph-mon[27906]: Cluster is now healthy 2024-04-26T02:09:05.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:05 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:09:05.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:05 smithi037 ceph-mon[27906]: fsmap 12 up:standby 2024-04-26T02:09:05.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:05 smithi045 ceph-mon[32520]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:09:05.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:05 smithi045 ceph-mon[32520]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:09:05.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:05 smithi045 ceph-mon[32520]: Cluster is now healthy 2024-04-26T02:09:05.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:05 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:09:05.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:05 smithi045 ceph-mon[32520]: fsmap 12 up:standby 2024-04-26T02:09:05.528 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:09:05.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:05 smithi071 ceph-mon[32578]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T02:09:05.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:05 smithi071 ceph-mon[32578]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T02:09:05.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:05 smithi071 ceph-mon[32578]: Cluster is now healthy 2024-04-26T02:09:05.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:05 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T02:09:05.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:05 smithi071 ceph-mon[32578]: fsmap 12 up:standby 2024-04-26T02:09:06.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:06 smithi037 ceph-mon[27906]: pgmap v346: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:06.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:06 smithi045 ceph-mon[32520]: pgmap v346: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:06.554 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:06 smithi071 ceph-mon[32578]: pgmap v346: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:07.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 -- 172.21.15.37:0/2599479144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3406b070 msgr2=0x7fac3410efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:07.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 --2- 172.21.15.37:0/2599479144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3406b070 0x7fac3410efb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fac1c0098f0 tx=0x7fac1c009c00 comp rx=0 tx=0).stop 2024-04-26T02:09:07.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 -- 172.21.15.37:0/2599479144 shutdown_connections 2024-04-26T02:09:07.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 --2- 172.21.15.37:0/2599479144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac3406a750 0x7fac3406ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:07.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 --2- 172.21.15.37:0/2599479144 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3410f4f0 0x7fac341118e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:07.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 --2- 172.21.15.37:0/2599479144 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3406b070 0x7fac3410efb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:07.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 -- 172.21.15.37:0/2599479144 >> 172.21.15.37:0/2599479144 conn(0x7fac340788f0 msgr2=0x7fac34078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:07.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 -- 172.21.15.37:0/2599479144 shutdown_connections 2024-04-26T02:09:07.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.791+0000 7fac3a400700 1 -- 172.21.15.37:0/2599479144 wait complete. 2024-04-26T02:09:07.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.792+0000 7fac3a400700 1 Processor -- start 2024-04-26T02:09:07.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.792+0000 7fac3a400700 1 -- start start 2024-04-26T02:09:07.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac3a400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac3406a750 0x7fac34103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:07.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac3a400700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3406b070 0x7fac34104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:07.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac3a400700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 0x7fac341046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac3a400700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fac34114e80 con 0x7fac3406a750 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac3a400700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7fac34114d00 con 0x7fac3406b070 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac3a400700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7fac34115000 con 0x7fac3410f4f0 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 0x7fac341046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33400700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac3406a750 0x7fac34103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 0x7fac341046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.71:3300/0 says I am v2:172.21.15.37:59108/0 (socket says 172.21.15.37:59108) 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 -- 172.21.15.37:0/1353298089 learned_addr learned my addr 172.21.15.37:0/1353298089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac2aa00700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3406b070 0x7fac34104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 -- 172.21.15.37:0/1353298089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3406b070 msgr2=0x7fac34104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3406b070 0x7fac34104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 -- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac3406a750 msgr2=0x7fac34103c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac3406a750 0x7fac34103c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:07.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.793+0000 7fac33e00700 1 -- 172.21.15.37:0/1353298089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac341abac0 con 0x7fac3410f4f0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.794+0000 7fac33e00700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 0x7fac341046a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00efc0 tx=0x7fac2c00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.794+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c0092e0 con 0x7fac3410f4f0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.794+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac2c004bb0 con 0x7fac3410f4f0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.794+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac2c004020 con 0x7fac3410f4f0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.794+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac341abd50 con 0x7fac3410f4f0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.794+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac341ac210 con 0x7fac3410f4f0 2024-04-26T02:09:07.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.795+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac34101f60 con 0x7fac3410f4f0 2024-04-26T02:09:07.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.798+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fac2c01b030 con 0x7fac3410f4f0 2024-04-26T02:09:07.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.799+0000 7fac30c00700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fac180a3e30 0x7fac180a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:07.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.799+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8555+0+0 (secure 0 0 0) 0x7fac2c0d0190 con 0x7fac3410f4f0 2024-04-26T02:09:07.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.799+0000 7fac33400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fac180a3e30 0x7fac180a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:07.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.800+0000 7fac33400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fac180a3e30 0x7fac180a62f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac24005fd0 tx=0x7fac24009310 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:09:07.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:07.803+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fac2c022020 con 0x7fac3410f4f0 2024-04-26T02:09:08.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.037+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fac3402e140 con 0x7fac3410f4f0 2024-04-26T02:09:08.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.183+0000 7fac30c00700 1 -- 172.21.15.37:0/1353298089 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) ==== 181+0+0 (secure 0 0 0) 0x7fac2c010040 con 0x7fac3410f4f0 2024-04-26T02:09:08.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.188+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fac180a3e30 msgr2=0x7fac180a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:08.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.188+0000 7fac3a400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fac180a3e30 0x7fac180a62f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac24005fd0 tx=0x7fac24009310 comp rx=0 tx=0).stop 2024-04-26T02:09:08.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.188+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 msgr2=0x7fac341046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:08.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.188+0000 7fac3a400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 0x7fac341046a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fac2c00efc0 tx=0x7fac2c00c3a0 comp rx=0 tx=0).stop 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 shutdown_connections 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fac3406a750 0x7fac34103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7fac180a3e30 0x7fac180a62f0 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac24005fd0 tx=0x7fac24009310 comp rx=0 tx=0).stop 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7fac3410f4f0 0x7fac341046a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 --2- 172.21.15.37:0/1353298089 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7fac3406b070 0x7fac34104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 >> 172.21.15.37:0/1353298089 conn(0x7fac340788f0 msgr2=0x7fac341109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.189+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 shutdown_connections 2024-04-26T02:09:08.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:08.190+0000 7fac3a400700 1 -- 172.21.15.37:0/1353298089 wait complete. 2024-04-26T02:09:08.191 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-04-26T02:09:08.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:08 smithi037 ceph-mon[27906]: pgmap v347: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:08.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:08 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/1353298089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:08.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:08 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:08.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:08 smithi045 ceph-mon[32520]: pgmap v347: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:08.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:08 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/1353298089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:08.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:08 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:08.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:08 smithi071 ceph-mon[32578]: pgmap v347: 97 pgs: 97 active+clean; 2.7 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 0 B/s wr, 3 op/s 2024-04-26T02:09:08.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:08 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/1353298089' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:08.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:08 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:09.049 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T02:09:09.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:09 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:09:09.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:09 smithi037 ceph-mon[27906]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:09:09.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:09 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:09:09.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:09 smithi045 ceph-mon[32520]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:09:09.536 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T02:09:09.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:09 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:09:09.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:09 smithi071 ceph-mon[32578]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T02:09:10.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:10 smithi037 ceph-mon[27906]: pgmap v349: 65 pgs: 65 active+clean; 577 KiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:09:10.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:10 smithi045 ceph-mon[32520]: pgmap v349: 65 pgs: 65 active+clean; 577 KiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:09:10.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:10 smithi071 ceph-mon[32578]: pgmap v349: 65 pgs: 65 active+clean; 577 KiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:09:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 -- 172.21.15.37:0/2397581668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c84075590 msgr2=0x7f6c8410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 --2- 172.21.15.37:0/2397581668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c84075590 0x7f6c8410ef00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6c700098f0 tx=0x7f6c70009c00 comp rx=0 tx=0).stop 2024-04-26T02:09:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 -- 172.21.15.37:0/2397581668 shutdown_connections 2024-04-26T02:09:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 --2- 172.21.15.37:0/2397581668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c84075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 --2- 172.21.15.37:0/2397581668 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c8410f440 0x7f6c84111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:11.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 --2- 172.21.15.37:0/2397581668 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c84075590 0x7f6c8410ef00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:11.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.765+0000 7f6c8b800700 1 -- 172.21.15.37:0/2397581668 >> 172.21.15.37:0/2397581668 conn(0x7f6c84079ba0 msgr2=0x7f6c84079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:11.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.766+0000 7f6c8b800700 1 -- 172.21.15.37:0/2397581668 shutdown_connections 2024-04-26T02:09:11.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.766+0000 7f6c8b800700 1 -- 172.21.15.37:0/2397581668 wait complete. 2024-04-26T02:09:11.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.767+0000 7f6c8b800700 1 Processor -- start 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.767+0000 7f6c8b800700 1 -- start start 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c8b800700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c8419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c8b800700 1 --2- >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84075590 0x7f6c8419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c8b800700 1 --2- >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8410f440 0x7f6c8419fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c8b800700 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84114e00 con 0x7f6c84074c70 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c8b800700 1 -- --> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84114c80 con 0x7f6c8410f440 2024-04-26T02:09:11.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c8b800700 1 -- --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_getmap magic: 0 -- 0x7f6c84114f80 con 0x7f6c84075590 2024-04-26T02:09:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c88e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c8419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c88e00700 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c8419b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43440/0 (socket says 172.21.15.37:43440) 2024-04-26T02:09:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c88e00700 1 -- 172.21.15.37:0/915658355 learned_addr learned my addr 172.21.15.37:0/915658355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-04-26T02:09:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c83e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84075590 0x7f6c8419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:11.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.768+0000 7f6c89800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8410f440 0x7f6c8419fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c83e00700 1 -- 172.21.15.37:0/915658355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8410f440 msgr2=0x7f6c8419fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c83e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8410f440 0x7f6c8419fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c83e00700 1 -- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 msgr2=0x7f6c8419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c83e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c8419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c83e00700 1 -- 172.21.15.37:0/915658355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c841a0350 con 0x7f6c84075590 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c88e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c8419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c89800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8410f440 0x7f6c8419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c83e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84075590 0x7f6c8419b8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009230 tx=0x7f6c7000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c70005040 con 0x7f6c84075590 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c70005690 con 0x7f6c84075590 2024-04-26T02:09:11.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c700038e0 con 0x7f6c84075590 2024-04-26T02:09:11.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.769+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c841a05e0 con 0x7f6c84075590 2024-04-26T02:09:11.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.770+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c841a7ec0 con 0x7f6c84075590 2024-04-26T02:09:11.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.771+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c84076360 con 0x7f6c84075590 2024-04-26T02:09:11.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.773+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6c7000b760 con 0x7f6c84075590 2024-04-26T02:09:11.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.774+0000 7f6c81600700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6c6c0a3db0 0x7f6c6c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T02:09:11.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.774+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8030+0+0 (secure 0 0 0) 0x7f6c700d0c30 con 0x7f6c84075590 2024-04-26T02:09:11.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.774+0000 7f6c88e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6c6c0a3db0 0x7f6c6c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T02:09:11.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.775+0000 7f6c88e00700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6c6c0a3db0 0x7f6c6c0a6270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009750 tx=0x7f6c78009310 comp rx=0 tx=0).ready entity=mgr.14856 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T02:09:11.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:11.778+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c70013030 con 0x7f6c84075590 2024-04-26T02:09:12.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.018+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 --> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6c8419c6a0 con 0x7f6c84075590 2024-04-26T02:09:12.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.222+0000 7f6c81600700 1 -- 172.21.15.37:0/915658355 <== mon.1 v2:172.21.15.71:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) ==== 169+0+0 (secure 0 0 0) 0x7f6c7009d500 con 0x7f6c84075590 2024-04-26T02:09:12.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.229+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6c6c0a3db0 msgr2=0x7f6c6c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:12.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.229+0000 7f6c8b800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6c6c0a3db0 0x7f6c6c0a6270 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009750 tx=0x7f6c78009310 comp rx=0 tx=0).stop 2024-04-26T02:09:12.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.229+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84075590 msgr2=0x7f6c8419b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T02:09:12.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.229+0000 7f6c8b800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84075590 0x7f6c8419b8d0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6c70009230 tx=0x7f6c7000fa00 comp rx=0 tx=0).stop 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.230+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 shutdown_connections 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.230+0000 7f6c8b800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6c84074c70 0x7f6c8419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.230+0000 7f6c8b800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.37:6800/3717150907,v1:172.21.15.37:6801/3717150907] conn(0x7f6c6c0a3db0 0x7f6c6c0a6270 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.230+0000 7f6c8b800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.45:3300/0,v1:172.21.15.45:6789/0] conn(0x7f6c8410f440 0x7f6c8419fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.230+0000 7f6c8b800700 1 --2- 172.21.15.37:0/915658355 >> [v2:172.21.15.71:3300/0,v1:172.21.15.71:6789/0] conn(0x7f6c84075590 0x7f6c8419b8d0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.231+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 >> 172.21.15.37:0/915658355 conn(0x7f6c84079ba0 msgr2=0x7f6c840ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.231+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 shutdown_connections 2024-04-26T02:09:12.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-04-26T02:09:12.231+0000 7f6c8b800700 1 -- 172.21.15.37:0/915658355 wait complete. 2024-04-26T02:09:12.234 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-04-26T02:09:12.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:12 smithi037 ceph-mon[27906]: pgmap v350: 65 pgs: 65 active+clean; 577 KiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:09:12.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:12 smithi037 ceph-mon[27906]: from='client.? 172.21.15.37:0/915658355' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:12.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:12 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:12 smithi045 ceph-mon[32520]: pgmap v350: 65 pgs: 65 active+clean; 577 KiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:09:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:12 smithi045 ceph-mon[32520]: from='client.? 172.21.15.37:0/915658355' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:12.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:12 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:12.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:12 smithi071 ceph-mon[32578]: pgmap v350: 65 pgs: 65 active+clean; 577 KiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T02:09:12.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:12 smithi071 ceph-mon[32578]: from='client.? 172.21.15.37:0/915658355' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:12.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:12 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T02:09:13.171 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:09:13.269 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:09:13.303 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:09:13.336 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T02:09:13.336 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:09:13.395 DEBUG:teuthology.orchestra.run.smithi045:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:09:13.411 DEBUG:teuthology.orchestra.run.smithi071:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T02:09:13.429 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T02:09:13.429 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T02:09:13.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a 2024-04-26T02:09:13.459 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:13 smithi037 ceph-mon[27906]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:09:13.460 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:13 smithi037 ceph-mon[27906]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:09:13.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:13 smithi045 ceph-mon[32520]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:09:13.505 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:13 smithi045 ceph-mon[32520]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:09:13.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:13 smithi071 ceph-mon[32578]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T02:09:13.653 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:13 smithi071 ceph-mon[32578]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T02:09:13.959 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:13 smithi037 systemd[1]: Stopping Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:14.358 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a[27902]: 2024-04-26T02:09:13.966+0000 7f4351600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:14.358 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a[27902]: 2024-04-26T02:09:13.966+0000 7f4351600700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T02:09:14.358 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:14 smithi037 podman[94152]: 2024-04-26 02:09:14.111277645 +0000 UTC m=+0.290090073 container died d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T02:09:14.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:14 smithi037 podman[94152]: 2024-04-26 02:09:14.606296063 +0000 UTC m=+0.785108484 container cleanup d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T02:09:14.614 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:14 smithi037 bash[94152]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a 2024-04-26T02:09:15.066 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:14 smithi037 podman[94166]: 2024-04-26 02:09:14.957914472 +0000 UTC m=+0.823365947 container remove d2b78331dbde7bf6d37b6fc124730531b7c0ff2cb07b48483dd1250d82987233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-26T02:09:15.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T02:09:15.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:15 smithi037 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T02:09:15.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:15 smithi037 systemd[1]: Stopped Ceph mon.a for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T02:09:15.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Apr 26 02:09:15 smithi037 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.a.service: Consumed 44.762s CPU time. 2024-04-26T02:09:15.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:15.424 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T02:09:15.424 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T02:09:15.424 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.b 2024-04-26T02:09:15.733 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:15 smithi045 systemd[1]: Stopping Ceph mon.b for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:16.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-b[32516]: 2024-04-26T02:09:15.733+0000 7fcdaecf4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:16.005 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-b[32516]: 2024-04-26T02:09:15.733+0000 7fcdaecf4700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T02:09:16.006 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:15 smithi045 podman[69772]: 2024-04-26 02:09:15.836395109 +0000 UTC m=+0.205752324 container died f99f16de2f2b86f2e9bff8e36a64d0ed91a1db3a74db94daea5f66c208a0e7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T02:09:16.434 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:16 smithi045 podman[69772]: 2024-04-26 02:09:16.209324425 +0000 UTC m=+0.578681639 container cleanup f99f16de2f2b86f2e9bff8e36a64d0ed91a1db3a74db94daea5f66c208a0e7f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:09:16.434 INFO:journalctl@ceph.mon.b.smithi045.stdout:Apr 26 02:09:16 smithi045 bash[69772]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-b 2024-04-26T02:09:16.554 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T02:09:16.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:16.602 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T02:09:16.602 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T02:09:16.602 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.c 2024-04-26T02:09:16.894 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:16 smithi071 systemd[1]: Stopping Ceph mon.c for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:17.148 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c[32570]: 2024-04-26T02:09:16.893+0000 7fb2d968c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:17.149 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c[32570]: 2024-04-26T02:09:16.893+0000 7fb2d968c700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T02:09:17.403 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 podman[69293]: 2024-04-26 02:09:17.150285899 +0000 UTC m=+0.351395627 container died b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_CLEAN=True) 2024-04-26T02:09:17.686 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 podman[69293]: 2024-04-26 02:09:17.494358329 +0000 UTC m=+0.695468065 container cleanup b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-26T02:09:17.686 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 bash[69293]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c 2024-04-26T02:09:17.919 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T02:09:17.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 podman[69306]: 2024-04-26 02:09:17.687267161 +0000 UTC m=+0.532323162 container remove b141b1cd90a29c6f8bf1eb5f4b0362da21b535caa30e637d7f17f97fc4960ceb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T02:09:17.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.c.service: Deactivated successfully. 2024-04-26T02:09:17.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 systemd[1]: Stopped Ceph mon.c for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T02:09:17.940 INFO:journalctl@ceph.mon.c.smithi071.stdout:Apr 26 02:09:17 smithi071 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mon.c.service: Consumed 15.030s CPU time. 2024-04-26T02:09:17.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:17.966 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T02:09:17.966 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T02:09:17.966 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x 2024-04-26T02:09:18.459 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:18 smithi037 systemd[1]: Stopping Ceph mgr.x for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:18.960 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T02:09:18.593+0000 7ff3bf000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:18.960 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x[28139]: 2024-04-26T02:09:18.593+0000 7ff3bf000700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-26T02:09:18.960 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:18 smithi037 podman[94286]: 2024-04-26 02:09:18.6899306 +0000 UTC m=+0.285497081 container died 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:09:19.445 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:19 smithi037 podman[94286]: 2024-04-26 02:09:19.194135117 +0000 UTC m=+0.789701637 container cleanup 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-26T02:09:19.445 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:19 smithi037 bash[94286]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x 2024-04-26T02:09:19.709 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Apr 26 02:09:19 smithi037 podman[94300]: 2024-04-26 02:09:19.445500326 +0000 UTC m=+0.743318811 container remove 8472511c75539dc12def10ba41db87b1fc9947a36bc86f9e5e0446f481d31620 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:09:20.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T02:09:20.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:20.152 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T02:09:20.152 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T02:09:20.152 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.y 2024-04-26T02:09:20.461 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 02:09:20 smithi045 systemd[1]: Stopping Ceph mgr.y for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:20.755 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 02:09:20 smithi045 podman[69894]: 2024-04-26 02:09:20.460190935 +0000 UTC m=+0.115316603 container died 62f7b75aa845a2efc0c4d4a1c09fc0a0d12b7bc92cff3d3203a6982e778715f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-26T02:09:21.089 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 02:09:20 smithi045 podman[69894]: 2024-04-26 02:09:20.873028465 +0000 UTC m=+0.528154132 container cleanup 62f7b75aa845a2efc0c4d4a1c09fc0a0d12b7bc92cff3d3203a6982e778715f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T02:09:21.089 INFO:journalctl@ceph.mgr.y.smithi045.stdout:Apr 26 02:09:20 smithi045 bash[69894]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-y 2024-04-26T02:09:21.229 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T02:09:21.263 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:21.263 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T02:09:21.263 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T02:09:21.263 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.z 2024-04-26T02:09:21.584 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 02:09:21 smithi071 systemd[1]: Stopping Ceph mgr.z for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:21.864 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 02:09:21 smithi071 podman[69416]: 2024-04-26 02:09:21.585455607 +0000 UTC m=+0.133147991 container died 3f8112940c8a74dc3c42bab34ad5dca350d05bc355e86f38638cc9cae5f2bc31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T02:09:22.153 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 02:09:21 smithi071 podman[69416]: 2024-04-26 02:09:21.956831142 +0000 UTC m=+0.504523595 container cleanup 3f8112940c8a74dc3c42bab34ad5dca350d05bc355e86f38638cc9cae5f2bc31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T02:09:22.153 INFO:journalctl@ceph.mgr.z.smithi071.stdout:Apr 26 02:09:21 smithi071 bash[69416]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mgr-z 2024-04-26T02:09:22.371 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T02:09:22.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:22.404 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T02:09:22.404 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T02:09:22.404 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.0 2024-04-26T02:09:22.959 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:22 smithi037 systemd[1]: Stopping Ceph osd.0 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:23.459 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0[42574]: 2024-04-26T02:09:23.180+0000 7f9bef600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:23.459 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0[42574]: 2024-04-26T02:09:23.180+0000 7f9bef600700 -1 osd.0 103 *** Got signal Terminated *** 2024-04-26T02:09:23.459 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0[42574]: 2024-04-26T02:09:23.180+0000 7f9bef600700 -1 osd.0 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:09:28.959 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:28 smithi037 podman[94407]: 2024-04-26 02:09:28.639093457 +0000 UTC m=+5.739962401 container died 191eaf47060574c8da3feda6041216acdebc6c3c411c4e67cf51cc721bafc56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T02:09:29.266 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:29 smithi037 podman[94407]: 2024-04-26 02:09:29.115338947 +0000 UTC m=+6.216207884 container cleanup 191eaf47060574c8da3feda6041216acdebc6c3c411c4e67cf51cc721bafc56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:09:29.266 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:29 smithi037 bash[94407]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0 2024-04-26T02:09:29.709 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:29 smithi037 podman[94426]: 2024-04-26 02:09:29.2663078 +0000 UTC m=+0.616293907 container remove 191eaf47060574c8da3feda6041216acdebc6c3c411c4e67cf51cc721bafc56c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:30.459 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:30 smithi037 podman[94516]: 2024-04-26 02:09:30.035899887 +0000 UTC m=+0.033996261 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:09:30.459 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:30 smithi037 podman[94516]: 2024-04-26 02:09:30.192226513 +0000 UTC m=+0.190322923 container create 4c7d1db42740ffbc4a2bd362f4b01f8532454ae906d5e6f1ee69401148bd1e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:09:30.959 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:30 smithi037 podman[94516]: 2024-04-26 02:09:30.491252684 +0000 UTC m=+0.489349101 container init 4c7d1db42740ffbc4a2bd362f4b01f8532454ae906d5e6f1ee69401148bd1e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-26T02:09:30.959 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:30 smithi037 podman[94516]: 2024-04-26 02:09:30.503775333 +0000 UTC m=+0.501871766 container start 4c7d1db42740ffbc4a2bd362f4b01f8532454ae906d5e6f1ee69401148bd1e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T02:09:30.960 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:30 smithi037 podman[94516]: 2024-04-26 02:09:30.602863877 +0000 UTC m=+0.600960337 container attach 4c7d1db42740ffbc4a2bd362f4b01f8532454ae906d5e6f1ee69401148bd1e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:31.386 INFO:journalctl@ceph.osd.0.smithi037.stdout:Apr 26 02:09:30 smithi037 podman[94516]: 2024-04-26 02:09:30.975056316 +0000 UTC m=+0.973152733 container died 4c7d1db42740ffbc4a2bd362f4b01f8532454ae906d5e6f1ee69401148bd1e61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:31.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T02:09:31.894 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:31.895 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T02:09:31.895 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T02:09:31.895 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.1 2024-04-26T02:09:32.176 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:31 smithi045 systemd[1]: Stopping Ceph osd.1 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:32.505 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1[37139]: 2024-04-26T02:09:32.176+0000 7fb33ea40700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:32.505 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1[37139]: 2024-04-26T02:09:32.176+0000 7fb33ea40700 -1 osd.1 103 *** Got signal Terminated *** 2024-04-26T02:09:32.505 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1[37139]: 2024-04-26T02:09:32.176+0000 7fb33ea40700 -1 osd.1 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:09:37.827 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:37 smithi045 podman[70022]: 2024-04-26 02:09:37.545519559 +0000 UTC m=+5.454548502 container died dac5211d3b1a026f1e97b9f67f9a911e6ed7984ce6a98293b4d5360f78834b83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T02:09:38.177 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:37 smithi045 podman[70022]: 2024-04-26 02:09:37.927525899 +0000 UTC m=+5.836554854 container cleanup dac5211d3b1a026f1e97b9f67f9a911e6ed7984ce6a98293b4d5360f78834b83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:38.177 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:37 smithi045 bash[70022]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1 2024-04-26T02:09:38.505 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:38 smithi045 podman[70039]: 2024-04-26 02:09:38.178703372 +0000 UTC m=+0.631244080 container remove dac5211d3b1a026f1e97b9f67f9a911e6ed7984ce6a98293b4d5360f78834b83 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T02:09:38.842 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:38 smithi045 podman[70130]: 2024-04-26 02:09:38.506897892 +0000 UTC m=+0.109464499 container create af4c00909c39d1e35be80a3e0b7176409991e941459ec1e034c476eee0ea1f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T02:09:38.842 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:38 smithi045 podman[70130]: 2024-04-26 02:09:38.415839931 +0000 UTC m=+0.018406548 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:09:38.842 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:38 smithi045 podman[70130]: 2024-04-26 02:09:38.7512487 +0000 UTC m=+0.353815303 container init af4c00909c39d1e35be80a3e0b7176409991e941459ec1e034c476eee0ea1f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-26T02:09:38.842 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:38 smithi045 podman[70130]: 2024-04-26 02:09:38.754485269 +0000 UTC m=+0.357051872 container start af4c00909c39d1e35be80a3e0b7176409991e941459ec1e034c476eee0ea1f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T02:09:38.842 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:38 smithi045 podman[70130]: 2024-04-26 02:09:38.842851952 +0000 UTC m=+0.445418558 container attach af4c00909c39d1e35be80a3e0b7176409991e941459ec1e034c476eee0ea1f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T02:09:39.255 INFO:journalctl@ceph.osd.1.smithi045.stdout:Apr 26 02:09:39 smithi045 podman[70130]: 2024-04-26 02:09:39.061917385 +0000 UTC m=+0.664483994 container died af4c00909c39d1e35be80a3e0b7176409991e941459ec1e034c476eee0ea1f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:09:39.691 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T02:09:39.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:39.731 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T02:09:39.731 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T02:09:39.731 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.2 2024-04-26T02:09:40.041 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:39 smithi071 systemd[1]: Stopping Ceph osd.2 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:40.403 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2[37694]: 2024-04-26T02:09:40.041+0000 7f4048cf0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:40.403 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2[37694]: 2024-04-26T02:09:40.041+0000 7f4048cf0700 -1 osd.2 103 *** Got signal Terminated *** 2024-04-26T02:09:40.403 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2[37694]: 2024-04-26T02:09:40.041+0000 7f4048cf0700 -1 osd.2 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:09:45.800 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:45 smithi071 podman[69542]: 2024-04-26 02:09:45.482723353 +0000 UTC m=+5.551494195 container died e92327bef0269c9b1dcc81f4e02465bcf1950873f21cdbfc767a0011fd884953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-26T02:09:46.091 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:45 smithi071 podman[69542]: 2024-04-26 02:09:45.900315878 +0000 UTC m=+5.969086718 container cleanup e92327bef0269c9b1dcc81f4e02465bcf1950873f21cdbfc767a0011fd884953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:09:46.092 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:45 smithi071 bash[69542]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2 2024-04-26T02:09:46.403 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:46 smithi071 podman[69572]: 2024-04-26 02:09:46.093064477 +0000 UTC m=+0.611613966 container remove e92327bef0269c9b1dcc81f4e02465bcf1950873f21cdbfc767a0011fd884953 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T02:09:46.753 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:46 smithi071 podman[69665]: 2024-04-26 02:09:46.395368061 +0000 UTC m=+0.018376769 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:09:46.753 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:46 smithi071 podman[69665]: 2024-04-26 02:09:46.52758551 +0000 UTC m=+0.150594200 container create d3c990583a5ae69cf9d8324b28b30027c34ee776cdf3d7fa6c82b689872f6b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:09:46.753 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:46 smithi071 podman[69665]: 2024-04-26 02:09:46.754716761 +0000 UTC m=+0.377725442 container init d3c990583a5ae69cf9d8324b28b30027c34ee776cdf3d7fa6c82b689872f6b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-26T02:09:47.051 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:46 smithi071 podman[69665]: 2024-04-26 02:09:46.757875743 +0000 UTC m=+0.380884428 container start d3c990583a5ae69cf9d8324b28b30027c34ee776cdf3d7fa6c82b689872f6b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:47.052 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:46 smithi071 podman[69665]: 2024-04-26 02:09:46.838009232 +0000 UTC m=+0.461017940 container attach d3c990583a5ae69cf9d8324b28b30027c34ee776cdf3d7fa6c82b689872f6b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-26T02:09:47.347 INFO:journalctl@ceph.osd.2.smithi071.stdout:Apr 26 02:09:47 smithi071 podman[69665]: 2024-04-26 02:09:47.053590523 +0000 UTC m=+0.676599230 container died d3c990583a5ae69cf9d8324b28b30027c34ee776cdf3d7fa6c82b689872f6b72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-26T02:09:47.655 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T02:09:47.695 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:47.695 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T02:09:47.695 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T02:09:47.695 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.3 2024-04-26T02:09:48.209 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:47 smithi037 systemd[1]: Stopping Ceph osd.3 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:48.602 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3[49397]: 2024-04-26T02:09:48.331+0000 7fa274600700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:48.602 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3[49397]: 2024-04-26T02:09:48.331+0000 7fa274600700 -1 osd.3 103 *** Got signal Terminated *** 2024-04-26T02:09:48.602 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:48 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3[49397]: 2024-04-26T02:09:48.331+0000 7fa274600700 -1 osd.3 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:09:50.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:49.702+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:50.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:49 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:49.978+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:50.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:50.614+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:51.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:50.725+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:51.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:50.994+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:51.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:50 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:50.749+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:51.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:51.597+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:52.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:51.684+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:52.033 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:51 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:51.754+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:52.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:52 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:52.033+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:52.904 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:52.565+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:53.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:52.669+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:53.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:52 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:52.786+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:53.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:52 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:52.990+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:53.608 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:53.264+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:53.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:53.609+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:54.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:53 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:53.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:54.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:53 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:53.758+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:54.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:53 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:53.973+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:54.209 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:53 smithi037 podman[94621]: 2024-04-26 02:09:53.930950174 +0000 UTC m=+5.777020514 container died cb8124ef9eaf5e886cf355f65ff9cf08ede4ab137769bf6a2fc126633adfe242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T02:09:54.632 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:54.258+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:54.702 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:54 smithi037 podman[94621]: 2024-04-26 02:09:54.35787073 +0000 UTC m=+6.203941081 container cleanup cb8124ef9eaf5e886cf355f65ff9cf08ede4ab137769bf6a2fc126633adfe242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:54.702 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:54 smithi037 bash[94621]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3 2024-04-26T02:09:54.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:54.633+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:54.959 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:54 smithi037 podman[94653]: 2024-04-26 02:09:54.700629891 +0000 UTC m=+0.754947758 container remove cb8124ef9eaf5e886cf355f65ff9cf08ede4ab137769bf6a2fc126633adfe242 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416) 2024-04-26T02:09:54.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:54.740+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:55.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:54 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:54.682+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:55.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:54 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:54.984+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:55.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:55.249+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:55.805 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:55 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:55.738+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:55.805 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:55 smithi037 podman[94741]: 2024-04-26 02:09:55.441480508 +0000 UTC m=+0.035280879 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:09:55.805 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:55 smithi037 podman[94741]: 2024-04-26 02:09:55.58487094 +0000 UTC m=+0.178671347 container create 3c721d1e45e5b2931f359f10f9c00849a22ec238ee2c87774cf6e1f08c1c68f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:56.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:55 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:55.677+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:56.153 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:55.663+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:56.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:56.020+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:56.209 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:55 smithi037 podman[94741]: 2024-04-26 02:09:55.905473605 +0000 UTC m=+0.499274019 container init 3c721d1e45e5b2931f359f10f9c00849a22ec238ee2c87774cf6e1f08c1c68f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T02:09:56.209 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:55 smithi037 podman[94741]: 2024-04-26 02:09:55.916882167 +0000 UTC m=+0.510682581 container start 3c721d1e45e5b2931f359f10f9c00849a22ec238ee2c87774cf6e1f08c1c68f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:09:56.209 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:56 smithi037 podman[94741]: 2024-04-26 02:09:56.029088239 +0000 UTC m=+0.622888696 container attach 3c721d1e45e5b2931f359f10f9c00849a22ec238ee2c87774cf6e1f08c1c68f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:09:56.645 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:56.282+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:56.695 INFO:journalctl@ceph.osd.3.smithi037.stdout:Apr 26 02:09:56 smithi037 podman[94741]: 2024-04-26 02:09:56.387259118 +0000 UTC m=+0.981059532 container died 3c721d1e45e5b2931f359f10f9c00849a22ec238ee2c87774cf6e1f08c1c68f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:09:56.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:56.647+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:56.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:56 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:56.779+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:57.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:56 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:56.716+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:57.100 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T02:09:57.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:09:57.222 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T02:09:57.222 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T02:09:57.223 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.4 2024-04-26T02:09:57.403 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:57.329+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:57.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:57 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:57.027+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:57.505 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:09:57 smithi045 systemd[1]: Stopping Ceph osd.4 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:09:57.861 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:09:57 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:09:57.548+0000 7f24953da700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:09:57.861 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:09:57 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:09:57.548+0000 7f24953da700 -1 osd.4 103 *** Got signal Terminated *** 2024-04-26T02:09:57.861 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:09:57 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:09:57.548+0000 7f24953da700 -1 osd.4 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:09:57.861 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:57 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:57.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:58.067 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:57 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:57.814+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:58.153 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:57.671+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:58.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:09:57 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:09:57.862+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:09:58.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:58 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:58.067+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:58.640 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:58.318+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:58.831 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:09:58 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:09:58.510+0000 7f24919ce700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.751762+0000 front 2024-04-26T02:09:36.751816+0000 (oldest deadline 2024-04-26T02:09:58.451972+0000) 2024-04-26T02:09:58.832 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:58 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:58.685+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:09:58.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:58.642+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:09:59.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:58 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:58.830+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:09:59.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:09:59 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:09:59.043+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:09:59.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:09:58 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:09:58.833+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:09:59.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:09:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:09:59.280+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:09:59.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:59.293+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:09:59.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:09:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:09:59.293+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:09:59.843 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:09:59 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:09:59.508+0000 7f24919ce700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.751762+0000 front 2024-04-26T02:09:36.751816+0000 (oldest deadline 2024-04-26T02:09:58.451972+0000) 2024-04-26T02:09:59.844 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:09:59 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:09:59.638+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:00.153 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:09:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:09:59.667+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:00.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:09:59 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:09:59.792+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:00.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:00.005+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:00.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:09:59 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:09:59.845+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:00.638 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:00.327+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:00.639 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:00.248+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:00.639 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:00.248+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:00.755 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:00 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:10:00.474+0000 7f24919ce700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.751762+0000 front 2024-04-26T02:09:36.751816+0000 (oldest deadline 2024-04-26T02:09:58.451972+0000) 2024-04-26T02:10:00.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:00 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:00.635+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:00.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:00.639+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:00.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:00.639+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:01.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:00.992+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:01.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:00.804+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:01.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:00 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:00.804+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:01.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:00 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:00.851+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:01.602 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:01 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:01.326+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:01.602 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:01 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:01.224+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:01.602 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:01 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:01.224+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:01.755 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:01 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:10:01.488+0000 7f24919ce700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.751762+0000 front 2024-04-26T02:09:36.751816+0000 (oldest deadline 2024-04-26T02:09:58.451972+0000) 2024-04-26T02:10:01.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:01 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:01.676+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:01.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:01 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:01.603+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:01.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:01 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:01.603+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:02.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:01.965+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:02.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:01.758+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:02.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:01 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:01.758+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:02.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:01 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:01.854+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:02.562 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:02 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:02.369+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:02.562 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:02 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:02.265+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:02.562 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:02 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:02.265+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:02.755 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:02 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4[44103]: 2024-04-26T02:10:02.468+0000 7f24919ce700 -1 osd.4 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.751762+0000 front 2024-04-26T02:09:36.751816+0000 (oldest deadline 2024-04-26T02:09:58.451972+0000) 2024-04-26T02:10:02.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:02 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:02.687+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:02.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:02 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:02.687+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:02.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:02 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:02.563+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:02.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:02 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:02.563+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:03.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:02 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:02.937+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:03.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:02 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:02.778+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:03.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:02 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:02.778+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:03.255 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:03 smithi045 podman[70247]: 2024-04-26 02:10:03.027379107 +0000 UTC m=+5.602356102 container died 6c0096b134203648ca0ba5b84a683ec7b2fd03cce3263efe7b2b0dd274648e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T02:10:03.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:02 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:02.853+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:03.523 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:03 smithi045 podman[70247]: 2024-04-26 02:10:03.348657676 +0000 UTC m=+5.923634677 container cleanup 6c0096b134203648ca0ba5b84a683ec7b2fd03cce3263efe7b2b0dd274648e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T02:10:03.523 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:03 smithi045 bash[70247]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4 2024-04-26T02:10:03.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:03 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:03.319+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:03.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:03 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:03.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:03.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:03 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:03.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:03.653 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:03 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:03.548+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:03.654 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:03 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:03.548+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:03.816 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:03 smithi045 podman[70265]: 2024-04-26 02:10:03.524829573 +0000 UTC m=+0.495809005 container remove 6c0096b134203648ca0ba5b84a683ec7b2fd03cce3263efe7b2b0dd274648e90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T02:10:03.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:03 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:03.734+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:03.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:03 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:03.734+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:04.095 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:03 smithi045 podman[70352]: 2024-04-26 02:10:03.875599236 +0000 UTC m=+0.018618846 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:04.096 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:03 smithi045 podman[70352]: 2024-04-26 02:10:03.975675975 +0000 UTC m=+0.118695643 container create dcc9a32c9c16c888758a34c2acc26f2a52fab20f8eebd93630fed726819f8212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:10:04.096 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:03 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:03.816+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:04.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:03.804+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:04.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:03.804+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:04.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:03.978+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:04.210 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:03 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:03.978+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:04.505 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:04 smithi045 podman[70352]: 2024-04-26 02:10:04.202926828 +0000 UTC m=+0.345946438 container init dcc9a32c9c16c888758a34c2acc26f2a52fab20f8eebd93630fed726819f8212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-26T02:10:04.505 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:04 smithi045 podman[70352]: 2024-04-26 02:10:04.206253387 +0000 UTC m=+0.349272996 container start dcc9a32c9c16c888758a34c2acc26f2a52fab20f8eebd93630fed726819f8212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T02:10:04.505 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:04 smithi045 podman[70352]: 2024-04-26 02:10:04.302916234 +0000 UTC m=+0.445935867 container attach dcc9a32c9c16c888758a34c2acc26f2a52fab20f8eebd93630fed726819f8212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:10:04.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:04 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:04.279+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:04.653 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:04 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:04.512+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:04.654 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:04 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:04.512+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:04.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:04 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:04.333+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:04.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:04 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:04.333+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:04.779 INFO:journalctl@ceph.osd.4.smithi045.stdout:Apr 26 02:10:04 smithi045 podman[70352]: 2024-04-26 02:10:04.510584274 +0000 UTC m=+0.653603891 container died dcc9a32c9c16c888758a34c2acc26f2a52fab20f8eebd93630fed726819f8212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T02:10:04.779 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:04 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:04.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:04.779 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:04 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:04.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:05.155 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:04 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:04.821+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:05.171 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T02:10:05.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:04.781+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:05.209 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:04.781+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:05.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:05.026+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:05.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:05.026+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:05.217 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:05.218 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T02:10:05.218 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T02:10:05.218 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.5 2024-04-26T02:10:05.508 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:05 smithi071 systemd[1]: Stopping Ceph osd.5 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:05.509 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:05.260+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:05.509 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:05.372+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:05.509 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:05.372+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:05.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:05.508+0000 7fbd71d0e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:05.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:05.508+0000 7fbd71d0e700 -1 osd.5 103 *** Got signal Terminated *** 2024-04-26T02:10:05.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:05.508+0000 7fbd71d0e700 -1 osd.5 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:05.904 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:05.517+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:05.904 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:05 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:05.517+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:06.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:05 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:05.836+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:06.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:05 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:05.720+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:06.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:05 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:05.720+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:06.074 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:05.810+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:06.074 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:05 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:05.810+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:06.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:06.074+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:06.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:06.074+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:06.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:06.074+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:06.549 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:06.235+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:06.549 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:06.235+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:06.550 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:06.346+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:06.550 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:06.346+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:06.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:06.550+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:06.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:06 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:06.550+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:07.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:06 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:06.848+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:07.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:06 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:06.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:07.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:06 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:06.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:07.034 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:06.781+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:07.034 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:06 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:06.781+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:07.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:07.034+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:07.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:07.034+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:07.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:07.034+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:07.596 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:07.279+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:07.597 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:07.279+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:07.597 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:07.349+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:07.597 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:07.349+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:07.597 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:07.349+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:07.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:07.598+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:07.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:07 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:07.598+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:08.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:07 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:07.866+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:08.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:07 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:07.717+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:08.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:07 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:07.717+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:08.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:07 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:07.717+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:08.015 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:07.740+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:08.015 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:07.740+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:08.015 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:07.740+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:08.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:08 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:08.015+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:08.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:08 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:08.015+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:08.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:08 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:08.015+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:08.619 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:08.307+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:08.620 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:08.307+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:08.620 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:08.358+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:08.620 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:08.358+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:08.620 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:08.358+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:08.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:08.620+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:08.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:08.620+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:08.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:08 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:08.620+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.140568+0000 front 2024-04-26T02:09:43.140693+0000 (oldest deadline 2024-04-26T02:10:08.440688+0000) 2024-04-26T02:10:09.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:08 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:08.857+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:09.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:08 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:08.703+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:09.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:08 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:08.703+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:09.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:08 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:08.703+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:09.056 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:08 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:08.754+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:09.056 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:08 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:08.754+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:09.056 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:08 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:08.754+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:09.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:09 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:09.056+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:09.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:09 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:09.056+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:09.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:09 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:09.056+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:09.596 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:09.320+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:09.596 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:09.320+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:09.596 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:09.371+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:09.596 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:09.371+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:09.596 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:09.371+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:09.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:09.597+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.438047+0000 front 2024-04-26T02:09:27.438064+0000 (oldest deadline 2024-04-26T02:09:50.338022+0000) 2024-04-26T02:10:09.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:09.597+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:33.238992+0000 front 2024-04-26T02:09:33.238967+0000 (oldest deadline 2024-04-26T02:10:00.239683+0000) 2024-04-26T02:10:09.903 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:09 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5[44366]: 2024-04-26T02:10:09.597+0000 7fbd6e302700 -1 osd.5 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.140568+0000 front 2024-04-26T02:09:43.140693+0000 (oldest deadline 2024-04-26T02:10:08.440688+0000) 2024-04-26T02:10:10.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:09.839+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:10.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:09.749+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:10.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:09.749+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:10.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:09 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:09.749+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:10.059 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:09 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:09.716+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:10.059 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:09 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:09.716+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:10.059 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:09 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:09.716+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:10.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:10 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:10.059+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:10.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:10 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:10.059+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:10.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:10 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:10.059+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:10.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:10.274+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:10.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:10.274+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:10.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:10.337+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:10.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:10.337+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:10.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:10 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:10.337+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:10.960 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:10 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:10.685+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:10.960 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:10 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:10.685+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:10.960 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:10 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:10.685+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:11.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:10 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:10.849+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:11.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:10 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:10.849+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:11.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:10 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:10.737+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:11.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:10 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:10.737+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:11.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:10 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:10.737+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:11.225 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:10 smithi071 podman[69772]: 2024-04-26 02:10:10.962762184 +0000 UTC m=+5.559813754 container died 80a7b09db79d51b4c25d1775c0eda7abbf3c4150f6985a84437b5ff42e4f941d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:10:11.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:11.062+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:11.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:11.063+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:11.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:11.063+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:11.608 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:11.225+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:11.609 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:11.225+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:11.609 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:11 smithi071 podman[69772]: 2024-04-26 02:10:11.325227462 +0000 UTC m=+5.922279024 container cleanup 80a7b09db79d51b4c25d1775c0eda7abbf3c4150f6985a84437b5ff42e4f941d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-26T02:10:11.609 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:11 smithi071 bash[69772]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5 2024-04-26T02:10:11.609 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:11.385+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:11.609 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:11.385+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:11.609 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:11 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:11.385+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:11.899 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:11 smithi071 podman[69790]: 2024-04-26 02:10:11.60995001 +0000 UTC m=+0.644076422 container remove 80a7b09db79d51b4c25d1775c0eda7abbf3c4150f6985a84437b5ff42e4f941d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-26T02:10:11.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:11.640+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:11.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:11.640+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:11.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:11.640+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:12.153 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:11 smithi071 podman[69887]: 2024-04-26 02:10:11.862986325 +0000 UTC m=+0.018110338 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:12.153 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:11 smithi071 podman[69887]: 2024-04-26 02:10:11.994454419 +0000 UTC m=+0.149578457 container create bb8c5fe11c74e7abbbd36b029dd0503f2d53b46b0e1c3ae2f7071c68d50f3479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-26T02:10:12.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:11.828+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:12.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:11.828+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:12.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:11.779+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:12.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:11.779+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:12.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:11 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:11.779+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:12.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:12 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:12.110+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:12.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:12 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:12.110+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:12.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:12 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:12.110+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:12.557 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:12.272+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:12.557 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:12.272+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:12.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:12.416+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:12.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:12.416+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:12.557 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:12 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:12.416+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:12.557 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:12 smithi071 podman[69887]: 2024-04-26 02:10:12.263592677 +0000 UTC m=+0.418716670 container init bb8c5fe11c74e7abbbd36b029dd0503f2d53b46b0e1c3ae2f7071c68d50f3479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T02:10:12.558 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:12 smithi071 podman[69887]: 2024-04-26 02:10:12.266816067 +0000 UTC m=+0.421940063 container start bb8c5fe11c74e7abbbd36b029dd0503f2d53b46b0e1c3ae2f7071c68d50f3479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-26T02:10:12.558 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:12 smithi071 podman[69887]: 2024-04-26 02:10:12.363496202 +0000 UTC m=+0.518620196 container attach bb8c5fe11c74e7abbbd36b029dd0503f2d53b46b0e1c3ae2f7071c68d50f3479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-26T02:10:12.558 INFO:journalctl@ceph.osd.5.smithi071.stdout:Apr 26 02:10:12 smithi071 podman[69887]: 2024-04-26 02:10:12.558220338 +0000 UTC m=+0.713344336 container died bb8c5fe11c74e7abbbd36b029dd0503f2d53b46b0e1c3ae2f7071c68d50f3479 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T02:10:12.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:12 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:12.612+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:12.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:12 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:12.612+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:12.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:12 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:12.612+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:13.142 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T02:10:13.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:13.180 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T02:10:13.180 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T02:10:13.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.6 2024-04-26T02:10:13.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:12 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:12.848+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:13.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:12 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:12.848+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:13.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:12 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:12.817+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:13.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:12 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:12.817+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:13.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:12 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:12.817+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:13.403 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:13 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:13.287+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:13.403 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:13 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:13.287+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:13.459 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 systemd[1]: Stopping Ceph osd.6 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:13.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:13.158+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:13.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:13.158+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:13.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:13.158+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:13.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:13 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:13.449+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:13.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:13 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:13.449+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:13.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:13 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:13.449+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:13.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:13.612+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:13.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:13.612+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:13.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:13.612+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:13.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:13.724+0000 7ff8a9400700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:13.960 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:13.724+0000 7ff8a9400700 -1 osd.6 103 *** Got signal Terminated *** 2024-04-26T02:10:13.960 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:13 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:13.724+0000 7ff8a9400700 -1 osd.6 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:14.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:13.894+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:14.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:13.894+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:14.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:13.784+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:14.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:13.784+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:14.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:13 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:13.784+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:14.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:14 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:14.159+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:14.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:14 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:14.159+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:14.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:14 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:14.159+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:14.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:14 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:14.312+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:14.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:14 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:14.312+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:14.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:14 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:14.410+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:14.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:14 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:14.410+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:14.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:14 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:14.410+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:14.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:14 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:14.655+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:14.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:14 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:14.655+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:14.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:14 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:14.655+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:15.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:14.872+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:15.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:14.872+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:15.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:14.800+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:15.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:14.800+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:15.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:14.800+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:15.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:14 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:14.800+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:15.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:15.205+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:15.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:15.205+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:15.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:15.205+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:15.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:15 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:15.284+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:15.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:15 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:15.284+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:15.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:15 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:15.414+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:15.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:15 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:15.414+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:15.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:15 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:15.414+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:15.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:15.645+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:15.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:15.645+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:15.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:15.645+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:16.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:15.908+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:16.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:15.908+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:16.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:15.908+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:16.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:15.780+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:16.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:15.780+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:16.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:15.780+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:16.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:15 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:15.780+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:16.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:16.288+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:16.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:16.288+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:16.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:16.375+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:16.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:16.375+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:16.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:16 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:16.375+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:16.654 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:16.251+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:16.654 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:16.251+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:16.654 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:16.251+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:16.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:16.654+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:16.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:16.654+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:16.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:16 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:16.654+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:17.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:16.953+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:17.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:16.953+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:17.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:16.953+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:17.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:16.766+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:17.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:16.766+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:17.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:16.766+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:17.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:16 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:16.766+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:17.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:17 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:17.276+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:17.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:17 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:17.276+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:17.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:17 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:17.413+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:17.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:17 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:17.413+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:17.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:17 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:17.413+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:17.661 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:17.249+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:17.662 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:17.249+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:17.662 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:17.249+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:17.662 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:17.249+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:17.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:17.661+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:17.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:17.661+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:17.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:17 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:17.661+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:18.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:17.910+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:18.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:17.910+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:18.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:17.910+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:18.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:17.796+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:18.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:17.796+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:18.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:17.796+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:18.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:17 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:17.796+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:18.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:18.201+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:18.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:18.201+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:18.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:18.201+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:18.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:18.201+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:18.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:18 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:18.293+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:18.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:18 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:18.293+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:18.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:18 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:18.372+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:18.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:18 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:18.372+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:18.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:18 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:18.372+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:18.856 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:18.635+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:27.029682+0000 front 2024-04-26T02:09:27.029237+0000 (oldest deadline 2024-04-26T02:09:50.529396+0000) 2024-04-26T02:10:18.856 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:18.635+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.832476+0000 front 2024-04-26T02:09:35.832365+0000 (oldest deadline 2024-04-26T02:10:00.532624+0000) 2024-04-26T02:10:18.856 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:18.635+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:43.336092+0000 front 2024-04-26T02:09:43.336356+0000 (oldest deadline 2024-04-26T02:10:06.835921+0000) 2024-04-26T02:10:18.856 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:18 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6[55897]: 2024-04-26T02:10:18.635+0000 7ff89fc00700 -1 osd.6 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:53.239892+0000 front 2024-04-26T02:09:53.239966+0000 (oldest deadline 2024-04-26T02:10:18.540214+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:18.759+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:18.759+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:18.759+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:18.759+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:18.866+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:18.866+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:19.157 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:18.866+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:19.459 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:19 smithi037 podman[94848]: 2024-04-26 02:10:19.318804095 +0000 UTC m=+5.718934338 container died 62d3a9a2862724bd70ed1e87d4ca15e3d15ac093b8cf48144d28def82103044e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:10:19.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:19 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:19.232+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:19.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:19 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:19.232+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:19.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:19 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:19.232+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:19.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:19 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:19.232+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:19.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:19.368+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:19.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:19.368+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:19.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:19.368+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:19.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:19.289+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:19.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:19.289+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:19.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:19 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:19.289+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:19.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:19 smithi037 podman[94848]: 2024-04-26 02:10:19.767903555 +0000 UTC m=+6.168033824 container cleanup 62d3a9a2862724bd70ed1e87d4ca15e3d15ac093b8cf48144d28def82103044e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T02:10:19.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:19 smithi037 bash[94848]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6 2024-04-26T02:10:20.230 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:19 smithi037 podman[94876]: 2024-04-26 02:10:19.960874844 +0000 UTC m=+0.634709347 container remove 62d3a9a2862724bd70ed1e87d4ca15e3d15ac093b8cf48144d28def82103044e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T02:10:20.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:19.827+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:20.256 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:19.827+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:20.256 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:19.827+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:20.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:19.760+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:20.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:19.760+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:20.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:19.760+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:20.256 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:19 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:19.760+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:20.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:20 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:20.360+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:20.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:20 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:20.360+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:20.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:20 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:20.360+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:20.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:20 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:20.281+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:20.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:20 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:20.281+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:20.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:20 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:20.281+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:20.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:20 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:20.230+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:20.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:20 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:20.231+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:20.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:20 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:20.231+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:20.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:20 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:20.231+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:20.978 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:20 smithi037 podman[94966]: 2024-04-26 02:10:20.632343141 +0000 UTC m=+0.038824979 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:20.979 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:20 smithi037 podman[94966]: 2024-04-26 02:10:20.744813973 +0000 UTC m=+0.151295844 container create ecfc2ce488e4cbea1c6d65db9c626ea4ed7fa48c984bfdbff154c77ce36466c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:10:21.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:20.792+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:21.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:20.792+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:21.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:20.793+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:21.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:20.735+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:21.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:20.735+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:21.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:20.735+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:21.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:20 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:20.735+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:21.460 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:21 smithi037 podman[94966]: 2024-04-26 02:10:21.065252629 +0000 UTC m=+0.471734506 container init ecfc2ce488e4cbea1c6d65db9c626ea4ed7fa48c984bfdbff154c77ce36466c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T02:10:21.460 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:21 smithi037 podman[94966]: 2024-04-26 02:10:21.079234157 +0000 UTC m=+0.485716008 container start ecfc2ce488e4cbea1c6d65db9c626ea4ed7fa48c984bfdbff154c77ce36466c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T02:10:21.460 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:21 smithi037 podman[94966]: 2024-04-26 02:10:21.208317371 +0000 UTC m=+0.614799272 container attach ecfc2ce488e4cbea1c6d65db9c626ea4ed7fa48c984bfdbff154c77ce36466c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T02:10:21.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:21 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:21.248+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:21.461 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:21 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:21.248+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:21.461 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:21 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:21.248+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:21.461 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:21 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:21.248+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:21.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:21 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:21.397+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:21.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:21 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:21.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:21.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:21 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:21.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:21.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:21 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:21.304+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:21.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:21 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:21.304+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:21.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:21 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:21.304+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:21.959 INFO:journalctl@ceph.osd.6.smithi037.stdout:Apr 26 02:10:21 smithi037 podman[94966]: 2024-04-26 02:10:21.556595904 +0000 UTC m=+0.963077782 container died ecfc2ce488e4cbea1c6d65db9c626ea4ed7fa48c984bfdbff154c77ce36466c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T02:10:22.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:21.779+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:22.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:21.779+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:22.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:21.779+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:22.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:21.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:22.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:21.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:22.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:21.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:22.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:21.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:22.392 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:22 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:22.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:22.392 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:22 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:22.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:22.392 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:22 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:22.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:22.393 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:22 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:22.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:22.440 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T02:10:22.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:22.562 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T02:10:22.562 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T02:10:22.563 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.7 2024-04-26T02:10:22.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:22 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:22.318+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:22.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:22 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:22.318+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:22.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:22 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:22.318+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:22.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:22 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:22.408+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:22.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:22 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:22.408+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:22.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:22 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:22.408+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:22.692+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:22.692+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:22.692+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:22.692+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 systemd[1]: Stopping Ceph osd.7 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:22.761+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:22.761+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:22.866 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:22.761+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:22.867 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:22.865+0000 7f0539067700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:22.867 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:22.865+0000 7f0539067700 -1 osd.7 103 *** Got signal Terminated *** 2024-04-26T02:10:22.867 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:22 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:22.865+0000 7f0539067700 -1 osd.7 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:23.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:23 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:23.321+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:23.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:23 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:23.321+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:23.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:23 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:23.321+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:23.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:23 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:23.448+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:23.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:23 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:23.448+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:23.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:23 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:23.448+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:23.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:23.279+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:23.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:23.279+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:23.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:23.279+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:23.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:23 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:23.279+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:24.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:23.769+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:24.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:23.769+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:24.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:23.769+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:24.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:23.711+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:24.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:23.711+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:24.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:23.711+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:24.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:23.711+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:24.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:23 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:23.711+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:24.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:24 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:24.361+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:24.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:24 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:24.361+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:24.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:24 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:24.361+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:24.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:24 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:24.462+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:24.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:24 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:24.462+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:24.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:24 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:24.462+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:24.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:24 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:24.322+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:24.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:24 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:24.322+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:24.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:24 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:24.322+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:24.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:24 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:24.322+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:25.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:24.753+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:25.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:24.753+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:25.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:24.753+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:25.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:24.753+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:01.358411+0000 front 2024-04-26T02:10:01.358341+0000 (oldest deadline 2024-04-26T02:10:24.258445+0000) 2024-04-26T02:10:25.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:24.665+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:25.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:24.665+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:25.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:24.665+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:25.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:24.666+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:25.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:24 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:24.666+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:25.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:25 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:25.370+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:25.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:25 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:25.370+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:25.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:25 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:25.370+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:25.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:25 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:25.415+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:25.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:25 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:25.415+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:25.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:25 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:25.415+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:25.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:25 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:25.319+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:25.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:25 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:25.319+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:25.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:25 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:25.319+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:25.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:25 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:25.319+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:26.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:25.797+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:26.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:25.797+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:26.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:25.797+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:26.006 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:25.797+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:01.358411+0000 front 2024-04-26T02:10:01.358341+0000 (oldest deadline 2024-04-26T02:10:24.258445+0000) 2024-04-26T02:10:26.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:25.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:26.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:25.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:26.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:25.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:26.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:25.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:26.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:25.691+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:26.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:26.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:26.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:26.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:26.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:26.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:26.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:26.417+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:26.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:26.417+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:26.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:26.417+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:26.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:26 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:26.417+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:26.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:26 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:26.336+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:26.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:26 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:26.336+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:26.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:26 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:26.336+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:26.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:26 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:26.336+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:27.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:26.815+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:27.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:26.815+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:26.815+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:26.815+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:01.358411+0000 front 2024-04-26T02:10:01.358341+0000 (oldest deadline 2024-04-26T02:10:24.258445+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:26.697+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:26.697+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:26.697+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:26.697+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:27.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:26 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:26.697+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:27.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:27.352+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:27.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:27.352+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:27.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:27.352+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:27.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:27.460+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:27.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:27.460+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:27.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:27.460+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:27.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:27 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:27.460+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:27.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:27 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:27.384+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:27.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:27 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:27.384+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:27.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:27 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:27.384+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:27.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:27 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:27.384+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:28.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:27.785+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.351670+0000 front 2024-04-26T02:09:34.351691+0000 (oldest deadline 2024-04-26T02:09:57.851959+0000) 2024-04-26T02:10:28.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:27.785+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.754216+0000 front 2024-04-26T02:09:44.754193+0000 (oldest deadline 2024-04-26T02:10:10.054379+0000) 2024-04-26T02:10:28.005 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:27.785+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.756823+0000 front 2024-04-26T02:09:52.756840+0000 (oldest deadline 2024-04-26T02:10:15.056594+0000) 2024-04-26T02:10:28.006 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7[51102]: 2024-04-26T02:10:27.785+0000 7f053565b700 -1 osd.7 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:01.358411+0000 front 2024-04-26T02:10:01.358341+0000 (oldest deadline 2024-04-26T02:10:24.258445+0000) 2024-04-26T02:10:28.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:27.740+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:28.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:27.741+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:28.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:27.741+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:28.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:27.741+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:28.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:27 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:27.741+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:28.505 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:28 smithi045 podman[70470]: 2024-04-26 02:10:28.209385725 +0000 UTC m=+5.440469451 container died 63f882b6abf37a7a65716f9289b8f57341c970be86780758f3628da44f382705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T02:10:28.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:28.506+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:28.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:28.506+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:28.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:28.506+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:28.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:28.506+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:28.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:28.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:28.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:28.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:28.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:28.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:28.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:28 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:28.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:28.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:28 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:28.335+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:28.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:28 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:28.335+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:28.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:28 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:28.335+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:28.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:28 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:28.335+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:28 smithi045 podman[70470]: 2024-04-26 02:10:28.532771393 +0000 UTC m=+5.763855108 container cleanup 63f882b6abf37a7a65716f9289b8f57341c970be86780758f3628da44f382705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5) 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:28 smithi045 bash[70470]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:28 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:28.726+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:28 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:28.726+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:28 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:28.726+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:28 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:28.726+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:28.816 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:28 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:28.726+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:29.255 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:28 smithi045 podman[70489]: 2024-04-26 02:10:28.817257502 +0000 UTC m=+0.604955070 container remove 63f882b6abf37a7a65716f9289b8f57341c970be86780758f3628da44f382705 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:10:29.611 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:29 smithi045 podman[70579]: 2024-04-26 02:10:29.209494254 +0000 UTC m=+0.018395556 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:29.611 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:29 smithi045 podman[70579]: 2024-04-26 02:10:29.326717365 +0000 UTC m=+0.135618655 container create e9ee869a13f909fa399ec8016735f0739d612cabc3f738f0e14d05d9b9c9905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T02:10:29.611 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:29 smithi045 podman[70579]: 2024-04-26 02:10:29.537516141 +0000 UTC m=+0.346417432 container init e9ee869a13f909fa399ec8016735f0739d612cabc3f738f0e14d05d9b9c9905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7-deactivate, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:10:29.611 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:29 smithi045 podman[70579]: 2024-04-26 02:10:29.540676738 +0000 UTC m=+0.349578031 container start e9ee869a13f909fa399ec8016735f0739d612cabc3f738f0e14d05d9b9c9905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T02:10:29.611 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:29 smithi045 podman[70579]: 2024-04-26 02:10:29.612330136 +0000 UTC m=+0.421231435 container attach e9ee869a13f909fa399ec8016735f0739d612cabc3f738f0e14d05d9b9c9905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:10:29.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:29.401+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:29.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:29.401+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:29.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:29.401+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:29.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:29.401+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:29.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:29.525+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:29.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:29.525+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:29.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:29.525+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:29.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:29 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:29.525+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:29.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:29 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:29.372+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:29.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:29 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:29.372+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:29.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:29 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:29.372+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:29.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:29 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:29.372+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:29.895 INFO:journalctl@ceph.osd.7.smithi045.stdout:Apr 26 02:10:29 smithi045 podman[70579]: 2024-04-26 02:10:29.855381588 +0000 UTC m=+0.664282881 container died e9ee869a13f909fa399ec8016735f0739d612cabc3f738f0e14d05d9b9c9905c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:10:29.896 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:29.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:29.896 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:29.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:29.896 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:29.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:29.896 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:29.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:29.896 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:29 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:29.704+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:30.482 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T02:10:30.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:30.519 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T02:10:30.520 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T02:10:30.520 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.8 2024-04-26T02:10:30.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:30.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:30.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:30.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:30.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 systemd[1]: Stopping Ceph osd.8 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:30.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:30.397+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:30.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:30.397+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:30.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:30.397+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:30.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:30.397+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:30.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:30 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:30.370+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:30.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:30 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:30.370+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:30.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:30 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:30.370+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:30.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:30 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:30.370+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:30.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:30.663+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:30.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:30.663+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:30.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:30.663+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:30.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:30.663+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:30.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:30.663+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:31.153 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.817+0000 7f7be2ac2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:31.153 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.817+0000 7f7be2ac2700 -1 osd.8 103 *** Got signal Terminated *** 2024-04-26T02:10:31.153 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:30 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:30.817+0000 7f7be2ac2700 -1 osd.8 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:31.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:31 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:31.321+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:31.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:31 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:31.321+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:31.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:31 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:31.321+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:31.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:31 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:31.321+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:31.903 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:31.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:31.903 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:31.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:31.903 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:31.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:31.903 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:31.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:31.904 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:31.570+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:07.822342+0000 front 2024-04-26T02:10:07.822298+0000 (oldest deadline 2024-04-26T02:10:31.322377+0000) 2024-04-26T02:10:31.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:31.411+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:31.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:31.411+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:31.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:31.411+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:31.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:31 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:31.411+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:32.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:31 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:31.657+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:32.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:31 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:31.657+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:32.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:31 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:31.657+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:32.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:31 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:31.657+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:32.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:31 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:31.657+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:32.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:32.538+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:32.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:32.538+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:32.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:32.538+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:32.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:32.538+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:32.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:32.538+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:07.822342+0000 front 2024-04-26T02:10:07.822298+0000 (oldest deadline 2024-04-26T02:10:31.322377+0000) 2024-04-26T02:10:32.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:32.365+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:32.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:32.365+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:32.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:32.365+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:32.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:32.365+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:32.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:32 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:32.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:32.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:32 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:32.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:32.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:32 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:32.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:32.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:32 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:32.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:32.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:32 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:32.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:33.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:32.618+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:33.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:32.618+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:33.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:32.618+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:33.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:32.618+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:33.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:32 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:32.618+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:33.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:33.335+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:33.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:33.335+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:33.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:33.335+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:33.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:33.335+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:33.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:33.497+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:33.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:33.497+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:33.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:33.497+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:33.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:33.497+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:33.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:33 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:33.497+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:07.822342+0000 front 2024-04-26T02:10:07.822298+0000 (oldest deadline 2024-04-26T02:10:31.322377+0000) 2024-04-26T02:10:33.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:33.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:33.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:33.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:33.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:33.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:33.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:33.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:33.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:33 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:33.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:34.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:33.623+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:34.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:33.623+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:34.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:33.623+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:34.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:33.623+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:34.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:33.623+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:34.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:33 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:33.623+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:34.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:34.500+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:34.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:34.500+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:34.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:34.500+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:34.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:34.500+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:34.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:34.500+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:07.822342+0000 front 2024-04-26T02:10:07.822298+0000 (oldest deadline 2024-04-26T02:10:31.322377+0000) 2024-04-26T02:10:34.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:34.383+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:34.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:34.383+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:34.655 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:34.383+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:34.655 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:34.383+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:34.655 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:34 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:34.383+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:34.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:34.315+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:34.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:34.315+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:34.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:34.315+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:34.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:34.315+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:34.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:34 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:34.315+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:35.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:34 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:34.590+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:35.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:34 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:34.590+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:35.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:34 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:34.590+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:35.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:34 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:34.590+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:35.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:34 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:34.590+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:35.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:34 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:34.590+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:35.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:35.457+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:35.116840+0000 front 2024-04-26T02:09:35.116899+0000 (oldest deadline 2024-04-26T02:09:58.616980+0000) 2024-04-26T02:10:35.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:35.457+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:41.518063+0000 front 2024-04-26T02:09:41.518039+0000 (oldest deadline 2024-04-26T02:10:06.218239+0000) 2024-04-26T02:10:35.653 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:35.457+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.519545+0000 front 2024-04-26T02:09:51.519636+0000 (oldest deadline 2024-04-26T02:10:19.119944+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:35.457+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:59.621254+0000 front 2024-04-26T02:09:59.621235+0000 (oldest deadline 2024-04-26T02:10:25.521350+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8[50933]: 2024-04-26T02:10:35.457+0000 7f7bdf0b6700 -1 osd.8 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:07.822342+0000 front 2024-04-26T02:10:07.822298+0000 (oldest deadline 2024-04-26T02:10:31.322377+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:35.403+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:35.403+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:35.403+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:35.403+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:35.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:35.403+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:35.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:35 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:35.323+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:35.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:35 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:35.323+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:35.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:35 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:35.323+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:35.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:35 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:35.323+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:35.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:35 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:35.323+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:36.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:35 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:35.547+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:36.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:35 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:35.547+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:36.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:35 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:35.547+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:36.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:35 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:35.547+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:36.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:35 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:35.547+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:36.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:35 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:35.547+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:36.467 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:36 smithi071 podman[69994]: 2024-04-26 02:10:36.185392138 +0000 UTC m=+5.466450515 container died ae247e50bb2ec61a6af42b7e0996cb10c761ec3e66ef60075bcdb47771acdfb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T02:10:36.467 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:36 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:36.431+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:36.467 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:36 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:36.431+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:36.467 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:36 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:36.431+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:36.467 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:36 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:36.431+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:36.468 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:36 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:36.431+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:36.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:36.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:36.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:36.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:36.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:36.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:36.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:36.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:36.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:36 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:36.295+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:36.792 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:36 smithi071 podman[69994]: 2024-04-26 02:10:36.567157162 +0000 UTC m=+5.848215546 container cleanup ae247e50bb2ec61a6af42b7e0996cb10c761ec3e66ef60075bcdb47771acdfb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:10:36.792 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:36 smithi071 bash[69994]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8 2024-04-26T02:10:37.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:36 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:36.515+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:37.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:36 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:36.515+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:37.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:36 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:36.515+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:37.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:36 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:36.515+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:37.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:36 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:36.515+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:37.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:36 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:36.515+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:37.117 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:36 smithi071 podman[70012]: 2024-04-26 02:10:36.793313905 +0000 UTC m=+0.603294861 container remove ae247e50bb2ec61a6af42b7e0996cb10c761ec3e66ef60075bcdb47771acdfb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-26T02:10:37.403 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:37 smithi071 podman[70106]: 2024-04-26 02:10:37.119021959 +0000 UTC m=+0.104671227 container create dc9400dfbc2111a80d222e26987092c566f4d4780cb92f2a177ca27b547802a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-26T02:10:37.403 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:37 smithi071 podman[70106]: 2024-04-26 02:10:37.03199721 +0000 UTC m=+0.017646486 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:37.403 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:37 smithi071 podman[70106]: 2024-04-26 02:10:37.354746328 +0000 UTC m=+0.340395592 container init dc9400dfbc2111a80d222e26987092c566f4d4780cb92f2a177ca27b547802a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T02:10:37.403 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:37 smithi071 podman[70106]: 2024-04-26 02:10:37.357684627 +0000 UTC m=+0.343333892 container start dc9400dfbc2111a80d222e26987092c566f4d4780cb92f2a177ca27b547802a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:10:37.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:37.258+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:37.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:37.258+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:37.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:37.258+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:37.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:37.258+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:37.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:37 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:37.258+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:37.903 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:37 smithi071 podman[70106]: 2024-04-26 02:10:37.434421637 +0000 UTC m=+0.420070917 container attach dc9400dfbc2111a80d222e26987092c566f4d4780cb92f2a177ca27b547802a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T02:10:37.903 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:37 smithi071 podman[70106]: 2024-04-26 02:10:37.648402598 +0000 UTC m=+0.634051857 container died dc9400dfbc2111a80d222e26987092c566f4d4780cb92f2a177ca27b547802a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:10:37.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:37 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:37.454+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:37.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:37 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:37.454+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:37.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:37 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:37.454+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:37.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:37 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:37.454+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:37.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:37 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:37.454+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:38.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:37 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:37.514+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:38.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:37 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:37.514+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:38.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:37 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:37.514+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:38.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:37 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:37.514+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:38.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:37 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:37.514+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:38.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:37 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:37.514+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:38.228 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T02:10:38.252 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:38 smithi071 podman[70132]: 2024-04-26 02:10:38.208651872 +0000 UTC m=+0.550585967 container remove dc9400dfbc2111a80d222e26987092c566f4d4780cb92f2a177ca27b547802a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-26T02:10:38.252 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:38 smithi071 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.8.service: Deactivated successfully. 2024-04-26T02:10:38.252 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:38 smithi071 systemd[1]: Stopped Ceph osd.8 for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T02:10:38.252 INFO:journalctl@ceph.osd.8.smithi071.stdout:Apr 26 02:10:38 smithi071 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.8.service: Consumed 27.042s CPU time. 2024-04-26T02:10:38.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:38.268 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T02:10:38.269 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T02:10:38.269 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.9 2024-04-26T02:10:38.642 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:38 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:38.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:38.643 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:38 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:38.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:38.643 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:38 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:38.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:38.643 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:38 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:38.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:38.643 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:38 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:38.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:38.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:38.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:38.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:38.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:38.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.290+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:38.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 systemd[1]: Stopping Ceph osd.9 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:38.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:38 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:38.470+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:38.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:38 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:38.470+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:38.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:38 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:38.470+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:38.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:38 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:38.470+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:38.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:38 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:38.470+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:38.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:38 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:38.470+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:39.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.859+0000 7faa16000700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:39.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.859+0000 7faa16000700 -1 osd.9 103 *** Got signal Terminated *** 2024-04-26T02:10:39.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:38 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:38.859+0000 7faa16000700 -1 osd.9 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:39.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:39 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:39.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:39.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:39 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:39.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:39.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:39 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:39.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:39.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:39 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:39.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:39.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:39 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:39.395+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:39.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:39.306+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:39.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:39.306+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:39.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:39.306+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:39.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:39.306+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:39.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:39 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:39.306+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:39.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:39 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:39.435+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:39.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:39 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:39.435+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:39.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:39 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:39.435+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:39.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:39 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:39.435+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:39.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:39 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:39.435+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:39.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:39 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:39.435+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:40.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:40.380+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:40.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:40.380+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:40.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:40.380+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:40.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:40.380+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:40.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:40.380+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:40.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:40.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:40.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:40.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:40.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:40.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:40.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:40.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:40.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:40 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:40.344+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:40.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:40 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:40.454+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:40.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:40 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:40.454+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:40.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:40 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:40.454+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:40.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:40 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:40.454+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:40.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:40 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:40.454+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:40.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:40 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:40.454+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:41.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:41.324+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:41.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:41.324+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:41.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:41.324+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:41.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:41.324+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:41.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:41.324+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:41.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:41 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:41.324+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.312468+0000 front 2024-04-26T02:10:17.312206+0000 (oldest deadline 2024-04-26T02:10:40.811972+0000) 2024-04-26T02:10:41.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:41.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:41.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:41.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:41.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:41.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:41.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:41 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:41.467+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:41.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:41 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:41.429+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:41.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:41 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:41.429+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:41.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:41 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:41.429+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:41.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:41 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:41.429+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:41.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:41 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:41.429+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:41.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:41 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:41.429+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:42.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:42.276+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:42.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:42.276+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:42.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:42.276+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:42.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:42.276+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:42.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:42.276+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:42.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:42 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:42.276+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.312468+0000 front 2024-04-26T02:10:17.312206+0000 (oldest deadline 2024-04-26T02:10:40.811972+0000) 2024-04-26T02:10:42.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:42 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:42.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:42.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:42 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:42.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:42.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:42 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:42.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:42.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:42 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:42.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:42.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:42 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:42.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:42.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:42 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:42.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:43.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:43.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:43.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:43.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:43.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:43.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:43.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:42 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:42.511+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:43.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:43.227+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:23.793109+0000 front 2024-04-26T02:09:23.793162+0000 (oldest deadline 2024-04-26T02:09:49.092850+0000) 2024-04-26T02:10:43.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:43.227+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:37.098516+0000 front 2024-04-26T02:09:37.098436+0000 (oldest deadline 2024-04-26T02:10:02.998643+0000) 2024-04-26T02:10:43.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:43.227+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.700875+0000 front 2024-04-26T02:09:44.700814+0000 (oldest deadline 2024-04-26T02:10:05.800806+0000) 2024-04-26T02:10:43.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:43.227+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:51.103211+0000 front 2024-04-26T02:09:51.103321+0000 (oldest deadline 2024-04-26T02:10:16.904404+0000) 2024-04-26T02:10:43.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:43.227+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.510050+0000 front 2024-04-26T02:10:08.510171+0000 (oldest deadline 2024-04-26T02:10:32.009955+0000) 2024-04-26T02:10:43.710 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:43 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9[62475]: 2024-04-26T02:10:43.227+0000 7faa0c800700 -1 osd.9 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.312468+0000 front 2024-04-26T02:10:17.312206+0000 (oldest deadline 2024-04-26T02:10:40.811972+0000) 2024-04-26T02:10:43.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:43.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:43.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:43.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:43.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:43.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:43.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:43 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:43.486+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:43.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:43 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:43.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:43.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:43 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:43.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:43.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:43 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:43.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:43.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:43 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:43.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:43.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:43 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:43.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:43.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:43 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:43.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:44.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:44 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:44.364+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:44.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:44 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:44.364+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:44.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:44 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:44.364+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:44.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:44 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:44.364+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:44.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:44 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:44.364+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:44.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:44 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:44.364+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:44.709 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:44 smithi037 podman[95070]: 2024-04-26 02:10:44.414283084 +0000 UTC m=+5.679278032 container died bf749144663e8e7a7ae9a6bbaf3aadb79675baa963e235e5600985075de9fa8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T02:10:44.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:44.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:44.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:44.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:44.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:44.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:44.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:44 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:44.500+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:45.203 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:44 smithi037 podman[95070]: 2024-04-26 02:10:44.985833488 +0000 UTC m=+6.250828497 container cleanup bf749144663e8e7a7ae9a6bbaf3aadb79675baa963e235e5600985075de9fa8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:10:45.203 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:44 smithi037 bash[95070]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9 2024-04-26T02:10:45.459 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:45 smithi037 podman[95089]: 2024-04-26 02:10:45.203376103 +0000 UTC m=+0.773408170 container remove bf749144663e8e7a7ae9a6bbaf3aadb79675baa963e235e5600985075de9fa8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416) 2024-04-26T02:10:45.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:45.382+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:45.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:45.382+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:45.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:45.382+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:45.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:45.382+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:45.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:45.382+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:45.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:45.382+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:45.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:45.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:45.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:45.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:45.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:45.755 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:45.756 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:45 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:45.505+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:46.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:46 smithi037 podman[95178]: 2024-04-26 02:10:46.034226613 +0000 UTC m=+0.038571112 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:46.460 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:46 smithi037 podman[95178]: 2024-04-26 02:10:46.162606256 +0000 UTC m=+0.166950786 container create bb5b259795b71ae81535b5db09376ac8ec5f39de953e4ee6abd1a970fbefc7fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:10:46.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:46 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:46.356+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:46.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:46 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:46.356+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:46.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:46 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:46.356+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:46.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:46 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:46.356+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:46.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:46 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:46.356+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:46.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:46 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:46.356+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:46.906 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:46 smithi037 podman[95178]: 2024-04-26 02:10:46.460012477 +0000 UTC m=+0.464357031 container init bb5b259795b71ae81535b5db09376ac8ec5f39de953e4ee6abd1a970fbefc7fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T02:10:46.906 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:46 smithi037 podman[95178]: 2024-04-26 02:10:46.470977745 +0000 UTC m=+0.475322279 container start bb5b259795b71ae81535b5db09376ac8ec5f39de953e4ee6abd1a970fbefc7fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-26T02:10:46.906 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:46 smithi037 podman[95178]: 2024-04-26 02:10:46.558728347 +0000 UTC m=+0.563072933 container attach bb5b259795b71ae81535b5db09376ac8ec5f39de953e4ee6abd1a970fbefc7fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-26T02:10:47.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:47.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:47.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:47.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:47.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:47.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:47.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:46 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:46.539+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:47.209 INFO:journalctl@ceph.osd.9.smithi037.stdout:Apr 26 02:10:46 smithi037 podman[95178]: 2024-04-26 02:10:46.963544943 +0000 UTC m=+0.967889452 container died bb5b259795b71ae81535b5db09376ac8ec5f39de953e4ee6abd1a970fbefc7fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:10:47.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:47.322+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:47.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:47.322+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:47.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:47.322+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:47.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:47.322+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:47.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:47.322+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:47.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:47 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:47.322+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:47.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T02:10:47.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:47.859 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T02:10:47.859 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T02:10:47.860 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.10 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:47.886 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:47.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:48.150 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:47 smithi045 systemd[1]: Stopping Ceph osd.10 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:48.505 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.151+0000 7fd457255700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:48.505 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.151+0000 7fd457255700 -1 osd.10 103 *** Got signal Terminated *** 2024-04-26T02:10:48.505 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.151+0000 7fd457255700 -1 osd.10 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:48.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:48 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:48.359+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:48.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:48 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:48.359+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:48.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:48 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:48.359+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:48.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:48 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:48.359+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:48.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:48 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:48.359+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:48.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:48 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:48.359+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:49.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:49.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:49.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:49.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:49.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:49.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:49.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:48 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:48.560+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:49.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:49.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:49.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:49.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:49.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:49.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:49.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:49.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:49.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:49.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:49.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:49 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:49.361+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:50.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:50.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:50.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:50.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:50.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:50.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:50.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:49 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:49.578+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:50.604 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:50.604 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:50.604 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:50.604 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:50.604 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:50.605 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:50.605 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:50 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:50.321+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:51.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:51.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:51.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:51.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:51.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:51.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:51.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:50 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:50.619+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:51.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:51.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:51.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:51.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:51.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:51.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:51.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:51 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:51.307+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:52.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:52.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:52.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:52.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:52.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:52.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:52.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:51 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:51.644+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:52.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:52.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:52.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:52.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:52.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:52.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:52.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:52 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:52.348+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:53.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:25.965660+0000 front 2024-04-26T02:09:25.965537+0000 (oldest deadline 2024-04-26T02:09:49.465451+0000) 2024-04-26T02:10:53.005 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:36.467357+0000 front 2024-04-26T02:09:36.467385+0000 (oldest deadline 2024-04-26T02:10:01.767565+0000) 2024-04-26T02:10:53.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:44.668647+0000 front 2024-04-26T02:09:44.668787+0000 (oldest deadline 2024-04-26T02:10:07.469055+0000) 2024-04-26T02:10:53.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-04-26T02:09:52.671099+0000 front 2024-04-26T02:09:52.671024+0000 (oldest deadline 2024-04-26T02:10:14.370882+0000) 2024-04-26T02:10:53.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:09:58.372537+0000 front 2024-04-26T02:09:58.372578+0000 (oldest deadline 2024-04-26T02:10:23.072656+0000) 2024-04-26T02:10:53.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:09.974953+0000 front 2024-04-26T02:10:09.975028+0000 (oldest deadline 2024-04-26T02:10:32.874785+0000) 2024-04-26T02:10:53.006 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:52 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10[58169]: 2024-04-26T02:10:52.668+0000 7fd453849700 -1 osd.10 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:12.875704+0000 front 2024-04-26T02:10:12.875570+0000 (oldest deadline 2024-04-26T02:10:41.076058+0000) 2024-04-26T02:10:53.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:53.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:53.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:53.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:53.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:53.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:53.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:53 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:53.398+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:53.984 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:53 smithi045 podman[70683]: 2024-04-26 02:10:53.614374096 +0000 UTC m=+5.556885480 container died d1b05e57991235c97c34f14eb19e675bf666d071c7fbd0f8bff8398effc3a722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-26T02:10:54.250 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:54 smithi045 podman[70683]: 2024-04-26 02:10:54.059209762 +0000 UTC m=+6.001721148 container cleanup d1b05e57991235c97c34f14eb19e675bf666d071c7fbd0f8bff8398effc3a722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T02:10:54.251 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:54 smithi045 bash[70683]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10 2024-04-26T02:10:54.505 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:54 smithi045 podman[70700]: 2024-04-26 02:10:54.251795851 +0000 UTC m=+0.624931470 container remove d1b05e57991235c97c34f14eb19e675bf666d071c7fbd0f8bff8398effc3a722 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:10:54.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:54.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:54.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:54.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:54.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:54.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:54.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:54 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:54.407+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:54.941 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:54 smithi045 podman[70794]: 2024-04-26 02:10:54.590853389 +0000 UTC m=+0.018416359 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:10:54.941 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:54 smithi045 podman[70794]: 2024-04-26 02:10:54.71089186 +0000 UTC m=+0.138454828 container create 27a103bdc4c70d16d862610ff19a062bf90d9aad8d324928e2f0ca498bb963cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-26T02:10:55.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 podman[70794]: 2024-04-26 02:10:55.038216919 +0000 UTC m=+0.465779887 container init 27a103bdc4c70d16d862610ff19a062bf90d9aad8d324928e2f0ca498bb963cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-26T02:10:55.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 podman[70794]: 2024-04-26 02:10:55.041419238 +0000 UTC m=+0.468982205 container start 27a103bdc4c70d16d862610ff19a062bf90d9aad8d324928e2f0ca498bb963cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:10:55.255 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 podman[70794]: 2024-04-26 02:10:55.121351784 +0000 UTC m=+0.548914751 container attach 27a103bdc4c70d16d862610ff19a062bf90d9aad8d324928e2f0ca498bb963cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-04-26T02:10:55.623 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 podman[70794]: 2024-04-26 02:10:55.34808823 +0000 UTC m=+0.775651189 container died 27a103bdc4c70d16d862610ff19a062bf90d9aad8d324928e2f0ca498bb963cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:10:55.854 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T02:10:55.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:55.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:55.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:55.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:55.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:55.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:55.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:55.434+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:55.917 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 podman[70819]: 2024-04-26 02:10:55.832902449 +0000 UTC m=+0.479391533 container remove 27a103bdc4c70d16d862610ff19a062bf90d9aad8d324928e2f0ca498bb963cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T02:10:55.917 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-26T02:10:55.917 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 systemd[1]: Stopped Ceph osd.10 for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T02:10:55.917 INFO:journalctl@ceph.osd.10.smithi045.stdout:Apr 26 02:10:55 smithi045 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.10.service: Consumed 33.349s CPU time. 2024-04-26T02:10:55.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:10:55.933 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T02:10:55.933 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T02:10:55.933 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.11 2024-04-26T02:10:56.226 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:55 smithi071 systemd[1]: Stopping Ceph osd.11 for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:10:56.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.226+0000 7f5d886a9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:10:56.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.226+0000 7f5d886a9700 -1 osd.11 103 *** Got signal Terminated *** 2024-04-26T02:10:56.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.226+0000 7f5d886a9700 -1 osd.11 103 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:56.654 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:56 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:56.461+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:57.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:57.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:57.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:57.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:57.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:57.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:57.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:57 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:57.430+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:58.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:58.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:58.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:58.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:58.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:58.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:58.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:58 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:58.466+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:10:59.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:10:59.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:10:59.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:10:59.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:10:59.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:10:59.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:10:59.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:10:59 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:10:59.424+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:11:00.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-04-26T02:09:26.501256+0000 front 2024-04-26T02:09:26.501100+0000 (oldest deadline 2024-04-26T02:09:52.401057+0000) 2024-04-26T02:11:00.903 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6804 osd.1 since back 2024-04-26T02:09:34.102263+0000 front 2024-04-26T02:09:34.102251+0000 (oldest deadline 2024-04-26T02:09:58.802172+0000) 2024-04-26T02:11:00.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6804 osd.2 since back 2024-04-26T02:09:42.303052+0000 front 2024-04-26T02:09:42.303251+0000 (oldest deadline 2024-04-26T02:10:06.903720+0000) 2024-04-26T02:11:00.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6812 osd.4 since back 2024-04-26T02:10:02.408445+0000 front 2024-04-26T02:10:02.408367+0000 (oldest deadline 2024-04-26T02:10:28.308218+0000) 2024-04-26T02:11:00.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.71:6812 osd.5 since back 2024-04-26T02:10:08.308718+0000 front 2024-04-26T02:10:08.308787+0000 (oldest deadline 2024-04-26T02:10:34.208843+0000) 2024-04-26T02:11:00.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-04-26T02:10:17.010717+0000 front 2024-04-26T02:10:17.010553+0000 (oldest deadline 2024-04-26T02:10:40.510359+0000) 2024-04-26T02:11:00.904 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:00 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11[57826]: 2024-04-26T02:11:00.439+0000 7f5d80c95700 -1 osd.11 103 heartbeat_check: no reply from 172.21.15.45:6820 osd.7 since back 2024-04-26T02:10:24.611534+0000 front 2024-04-26T02:10:24.611619+0000 (oldest deadline 2024-04-26T02:10:49.911313+0000) 2024-04-26T02:11:02.060 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:01 smithi071 podman[70225]: 2024-04-26 02:11:01.76793963 +0000 UTC m=+5.647857364 container died b73a5bd49aa28b4ebf2efc78b6cc0a7f3a4e845189360ff9a3f5d240a8d8542f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-26T02:11:02.360 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70225]: 2024-04-26 02:11:02.134836253 +0000 UTC m=+6.014753989 container cleanup b73a5bd49aa28b4ebf2efc78b6cc0a7f3a4e845189360ff9a3f5d240a8d8542f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T02:11:02.360 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 bash[70225]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11 2024-04-26T02:11:02.653 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70242]: 2024-04-26 02:11:02.360947098 +0000 UTC m=+0.596627077 container remove b73a5bd49aa28b4ebf2efc78b6cc0a7f3a4e845189360ff9a3f5d240a8d8542f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5) 2024-04-26T02:11:02.971 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70335]: 2024-04-26 02:11:02.575234847 +0000 UTC m=+0.017711502 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-26T02:11:02.971 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70335]: 2024-04-26 02:11:02.67821081 +0000 UTC m=+0.120687453 container create e524c5eac640417b43b1df640c422974cf2a8be68532c5d0d41b5b813aa9996f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T02:11:02.971 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70335]: 2024-04-26 02:11:02.897403322 +0000 UTC m=+0.339879962 container init e524c5eac640417b43b1df640c422974cf2a8be68532c5d0d41b5b813aa9996f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-26T02:11:02.971 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70335]: 2024-04-26 02:11:02.900540549 +0000 UTC m=+0.343017190 container start e524c5eac640417b43b1df640c422974cf2a8be68532c5d0d41b5b813aa9996f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T02:11:02.971 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:02 smithi071 podman[70335]: 2024-04-26 02:11:02.972253346 +0000 UTC m=+0.414730001 container attach e524c5eac640417b43b1df640c422974cf2a8be68532c5d0d41b5b813aa9996f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T02:11:03.403 INFO:journalctl@ceph.osd.11.smithi071.stdout:Apr 26 02:11:03 smithi071 podman[70335]: 2024-04-26 02:11:03.190716773 +0000 UTC m=+0.633193411 container died e524c5eac640417b43b1df640c422974cf2a8be68532c5d0d41b5b813aa9996f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:11:03.818 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T02:11:03.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:03.855 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T02:11:03.855 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T02:11:03.855 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.a 2024-04-26T02:11:04.209 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:03 smithi037 systemd[1]: Stopping Ceph mds.a for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:04.709 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a[69887]: 2024-04-26T02:11:04.428+0000 7f8fa1800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:04.709 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:04 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a[69887]: 2024-04-26T02:11:04.428+0000 7f8fa1800700 -1 mds.a *** got signal Terminated *** 2024-04-26T02:11:05.959 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:05 smithi037 podman[95293]: 2024-04-26 02:11:05.707230902 +0000 UTC m=+1.417350247 container died 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True) 2024-04-26T02:11:06.323 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:06 smithi037 podman[95293]: 2024-04-26 02:11:06.138528423 +0000 UTC m=+1.848647761 container cleanup 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:11:06.323 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:06 smithi037 bash[95293]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a 2024-04-26T02:11:06.710 INFO:journalctl@ceph.mds.a.smithi037.stdout:Apr 26 02:11:06 smithi037 podman[95306]: 2024-04-26 02:11:06.322968832 +0000 UTC m=+0.603119138 container remove 23296243e7ec2a1630a7e10b364472888bcc6fc0bccb8c78db535f517626cf8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-a, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T02:11:06.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T02:11:07.027 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:07.027 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T02:11:07.028 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T02:11:07.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.d 2024-04-26T02:11:07.459 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:07 smithi037 systemd[1]: Stopping Ceph mds.d for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:07.959 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d[68715]: 2024-04-26T02:11:07.648+0000 7f7d61800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:07.959 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:07 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d[68715]: 2024-04-26T02:11:07.648+0000 7f7d61800700 -1 mds.d *** got signal Terminated *** 2024-04-26T02:11:10.006 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:09 smithi037 podman[95413]: 2024-04-26 02:11:09.737315776 +0000 UTC m=+2.223148957 container died 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T02:11:10.343 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:10 smithi037 podman[95413]: 2024-04-26 02:11:10.158675433 +0000 UTC m=+2.644508624 container cleanup 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T02:11:10.343 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:10 smithi037 bash[95413]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d 2024-04-26T02:11:10.709 INFO:journalctl@ceph.mds.d.smithi037.stdout:Apr 26 02:11:10 smithi037 podman[95428]: 2024-04-26 02:11:10.342845886 +0000 UTC m=+0.587518979 container remove 5884dcd8bed9797890e953b41785f61d5043e88ff09f4b4f92e24e03b4e8d67a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-26T02:11:10.847 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T02:11:10.959 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:10.960 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T02:11:10.960 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T02:11:10.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.g 2024-04-26T02:11:11.459 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:11 smithi037 systemd[1]: Stopping Ceph mds.g for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:11.959 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g[69491]: 2024-04-26T02:11:11.616+0000 7faa52800700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:11.959 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:11 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g[69491]: 2024-04-26T02:11:11.616+0000 7faa52800700 -1 mds.g *** got signal Terminated *** 2024-04-26T02:11:13.459 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:13 smithi037 podman[95535]: 2024-04-26 02:11:13.176242529 +0000 UTC m=+1.725519381 container died 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-26T02:11:13.905 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:13 smithi037 podman[95535]: 2024-04-26 02:11:13.595574642 +0000 UTC m=+2.144851491 container cleanup 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-26T02:11:13.905 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:13 smithi037 bash[95535]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g 2024-04-26T02:11:14.162 INFO:journalctl@ceph.mds.g.smithi037.stdout:Apr 26 02:11:13 smithi037 podman[95549]: 2024-04-26 02:11:13.905204081 +0000 UTC m=+0.709838681 container remove 8f3b1f24b5a6874dd5b722ee69486a5dca834cdbc68f1587b614860ba2236122 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T02:11:14.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T02:11:14.293 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:14.293 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T02:11:14.293 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T02:11:14.293 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.j 2024-04-26T02:11:14.709 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:14 smithi037 systemd[1]: Stopping Ceph mds.j for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:15.459 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j[68289]: 2024-04-26T02:11:15.005+0000 7fcaa0200700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:15.459 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:15 smithi037 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j[68289]: 2024-04-26T02:11:15.005+0000 7fcaa0200700 -1 mds.j *** got signal Terminated *** 2024-04-26T02:11:16.959 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:16 smithi037 podman[95654]: 2024-04-26 02:11:16.70471075 +0000 UTC m=+1.880752773 container died 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-26T02:11:17.375 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:17 smithi037 podman[95654]: 2024-04-26 02:11:17.182803716 +0000 UTC m=+2.358845739 container cleanup 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:11:17.375 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:17 smithi037 bash[95654]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j 2024-04-26T02:11:17.709 INFO:journalctl@ceph.mds.j.smithi037.stdout:Apr 26 02:11:17 smithi037 podman[95667]: 2024-04-26 02:11:17.37515844 +0000 UTC m=+0.648505665 container remove 93f54a6229f252c82670e64f0640c7534e3802953373b62b89b591084dd8b474 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T02:11:17.891 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T02:11:18.001 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:18.001 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T02:11:18.001 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T02:11:18.001 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.b 2024-04-26T02:11:18.300 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:11:18 smithi045 systemd[1]: Stopping Ceph mds.b for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:18.755 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:11:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b[63673]: 2024-04-26T02:11:18.299+0000 7f77e8845700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:18.755 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:11:18 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b[63673]: 2024-04-26T02:11:18.299+0000 7f77e8845700 -1 mds.b *** got signal Terminated *** 2024-04-26T02:11:20.466 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:11:20 smithi045 podman[70898]: 2024-04-26 02:11:20.209437605 +0000 UTC m=+2.007902116 container died f1ba4643264ab94a5c01321acfaccf04204e9dba088da807cb35f89260e82249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T02:11:20.755 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:11:20 smithi045 podman[70898]: 2024-04-26 02:11:20.591997295 +0000 UTC m=+2.390461805 container cleanup f1ba4643264ab94a5c01321acfaccf04204e9dba088da807cb35f89260e82249 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-26T02:11:20.756 INFO:journalctl@ceph.mds.b.smithi045.stdout:Apr 26 02:11:20 smithi045 bash[70898]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-b 2024-04-26T02:11:20.981 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T02:11:21.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:21.016 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T02:11:21.016 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T02:11:21.016 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.e 2024-04-26T02:11:21.255 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:21 smithi045 systemd[1]: Stopping Ceph mds.e for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:21.755 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e[63039]: 2024-04-26T02:11:21.361+0000 7f15c428c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:21.755 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:21 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e[63039]: 2024-04-26T02:11:21.361+0000 7f15c428c700 -1 mds.e *** got signal Terminated *** 2024-04-26T02:11:24.845 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:24 smithi045 podman[71021]: 2024-04-26 02:11:24.584012549 +0000 UTC m=+3.349869471 container died 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True) 2024-04-26T02:11:25.112 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:24 smithi045 podman[71021]: 2024-04-26 02:11:24.920979139 +0000 UTC m=+3.686836077 container cleanup 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-26T02:11:25.112 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:24 smithi045 bash[71021]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e 2024-04-26T02:11:25.347 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T02:11:25.369 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:25 smithi045 podman[71047]: 2024-04-26 02:11:25.113315325 +0000 UTC m=+0.530555405 container remove 61f22fd4072f6c092e1cb2969c0207ed9f5bbf1b8ade57d015999a12d2400e17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T02:11:25.369 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:25 smithi045 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.e.service: Deactivated successfully. 2024-04-26T02:11:25.369 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:25 smithi045 systemd[1]: Stopped Ceph mds.e for 5d371fa8-036e-11ef-bc93-c7b262605968. 2024-04-26T02:11:25.369 INFO:journalctl@ceph.mds.e.smithi045.stdout:Apr 26 02:11:25 smithi045 systemd[1]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.e.service: Consumed 1.143s CPU time. 2024-04-26T02:11:25.381 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:25.382 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T02:11:25.382 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T02:11:25.382 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.h 2024-04-26T02:11:25.707 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 02:11:25 smithi045 systemd[1]: Stopping Ceph mds.h for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:26.005 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 02:11:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h[63348]: 2024-04-26T02:11:25.707+0000 7fc3a7ae8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:26.005 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 02:11:25 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h[63348]: 2024-04-26T02:11:25.707+0000 7fc3a7ae8700 -1 mds.h *** got signal Terminated *** 2024-04-26T02:11:29.358 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 02:11:29 smithi045 podman[71155]: 2024-04-26 02:11:29.087472728 +0000 UTC m=+3.480697053 container died a9f2e6a17d7b867d6236654ce8ee96c1b047cb66fdbae26c13046754d906698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:11:29.707 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 02:11:29 smithi045 podman[71155]: 2024-04-26 02:11:29.440704306 +0000 UTC m=+3.833928630 container cleanup a9f2e6a17d7b867d6236654ce8ee96c1b047cb66fdbae26c13046754d906698e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T02:11:29.707 INFO:journalctl@ceph.mds.h.smithi045.stdout:Apr 26 02:11:29 smithi045 bash[71155]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-h 2024-04-26T02:11:29.870 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T02:11:29.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:29.904 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T02:11:29.905 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T02:11:29.905 DEBUG:teuthology.orchestra.run.smithi045:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.k 2024-04-26T02:11:30.255 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:29 smithi045 systemd[1]: Stopping Ceph mds.k for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:30.256 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k[62730]: 2024-04-26T02:11:30.210+0000 7f4429b33700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:30.256 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:30 smithi045 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k[62730]: 2024-04-26T02:11:30.210+0000 7f4429b33700 -1 mds.k *** got signal Terminated *** 2024-04-26T02:11:31.543 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:31 smithi045 podman[71279]: 2024-04-26 02:11:31.27938495 +0000 UTC m=+1.154249231 container died a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-26T02:11:31.908 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:31 smithi045 podman[71279]: 2024-04-26 02:11:31.544127092 +0000 UTC m=+1.418991381 container cleanup a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:11:31.908 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:31 smithi045 bash[71279]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k 2024-04-26T02:11:31.908 INFO:journalctl@ceph.mds.k.smithi045.stdout:Apr 26 02:11:31 smithi045 podman[71294]: 2024-04-26 02:11:31.786877618 +0000 UTC m=+0.503526156 container remove a866d6b8464c1abb3a01e19a89e103c880a0ab85af92cb5f4fe70a55fe079fc0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T02:11:31.920 DEBUG:teuthology.orchestra.run.smithi045:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T02:11:31.953 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:31.953 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T02:11:31.953 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T02:11:31.953 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.c 2024-04-26T02:11:32.290 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 02:11:32 smithi071 systemd[1]: Stopping Ceph mds.c for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:32.653 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 02:11:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c[63011]: 2024-04-26T02:11:32.290+0000 7fda3606d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:32.653 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 02:11:32 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c[63011]: 2024-04-26T02:11:32.290+0000 7fda3606d700 -1 mds.c *** got signal Terminated *** 2024-04-26T02:11:34.632 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 02:11:34 smithi071 podman[70443]: 2024-04-26 02:11:34.354991963 +0000 UTC m=+2.185709554 container died 2f4077c0725f89d10bef708b587dca94f5243857c102596e43c11fa29ebab0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-26T02:11:34.903 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 02:11:34 smithi071 podman[70443]: 2024-04-26 02:11:34.70723238 +0000 UTC m=+2.537949981 container cleanup 2f4077c0725f89d10bef708b587dca94f5243857c102596e43c11fa29ebab0ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T02:11:34.904 INFO:journalctl@ceph.mds.c.smithi071.stdout:Apr 26 02:11:34 smithi071 bash[70443]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-c 2024-04-26T02:11:35.129 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T02:11:35.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:35.164 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T02:11:35.164 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T02:11:35.164 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.f 2024-04-26T02:11:35.653 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 02:11:35 smithi071 systemd[1]: Stopping Ceph mds.f for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:35.653 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 02:11:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f[62689]: 2024-04-26T02:11:35.485+0000 7f9dc8931700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:35.654 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 02:11:35 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f[62689]: 2024-04-26T02:11:35.485+0000 7f9dc8931700 -1 mds.f *** got signal Terminated *** 2024-04-26T02:11:39.769 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 02:11:39 smithi071 podman[70575]: 2024-04-26 02:11:39.492292464 +0000 UTC m=+4.114857626 container died 6d15fa8b32d8d41082f95a21f2ea8f946a5bf83fdaaf373fb235a62d0ac915a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-26T02:11:40.103 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 02:11:39 smithi071 podman[70575]: 2024-04-26 02:11:39.869539503 +0000 UTC m=+4.492104674 container cleanup 6d15fa8b32d8d41082f95a21f2ea8f946a5bf83fdaaf373fb235a62d0ac915a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-26T02:11:40.103 INFO:journalctl@ceph.mds.f.smithi071.stdout:Apr 26 02:11:39 smithi071 bash[70575]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-f 2024-04-26T02:11:40.234 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T02:11:40.265 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:40.265 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T02:11:40.265 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T02:11:40.265 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.i 2024-04-26T02:11:40.403 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:11:40 smithi071 systemd[1]: Stopping Ceph mds.i for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:40.906 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:11:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: 2024-04-26T02:11:40.581+0000 7fbf0421e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:40.906 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:11:40 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i[63315]: 2024-04-26T02:11:40.581+0000 7fbf0421e700 -1 mds.i *** got signal Terminated *** 2024-04-26T02:11:44.489 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:11:44 smithi071 podman[70695]: 2024-04-26 02:11:44.212250273 +0000 UTC m=+3.738382926 container died 3520e2016e1b6f625873a7e9aefea87577975f97f2349986ecf7cf37f89a9bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-26T02:11:44.857 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:11:44 smithi071 podman[70695]: 2024-04-26 02:11:44.572946857 +0000 UTC m=+4.099079516 container cleanup 3520e2016e1b6f625873a7e9aefea87577975f97f2349986ecf7cf37f89a9bea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T02:11:44.857 INFO:journalctl@ceph.mds.i.smithi071.stdout:Apr 26 02:11:44 smithi071 bash[70695]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-i 2024-04-26T02:11:44.990 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T02:11:45.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:45.023 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T02:11:45.023 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T02:11:45.023 DEBUG:teuthology.orchestra.run.smithi071:> sudo systemctl stop ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.l 2024-04-26T02:11:45.153 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 02:11:45 smithi071 systemd[1]: Stopping Ceph mds.l for 5d371fa8-036e-11ef-bc93-c7b262605968... 2024-04-26T02:11:45.653 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 02:11:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l[62378]: 2024-04-26T02:11:45.376+0000 7f1c67718700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T02:11:45.654 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 02:11:45 smithi071 ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l[62378]: 2024-04-26T02:11:45.376+0000 7f1c67718700 -1 mds.l *** got signal Terminated *** 2024-04-26T02:11:48.654 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 02:11:48 smithi071 podman[70815]: 2024-04-26 02:11:48.338799059 +0000 UTC m=+3.102924998 container died b8e9d2b16da61808a3e4a63193d48bfd31abce8d8e0f10f42cc84a6b747fd768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS) 2024-04-26T02:11:49.259 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 02:11:48 smithi071 podman[70815]: 2024-04-26 02:11:48.993063475 +0000 UTC m=+3.757189416 container cleanup b8e9d2b16da61808a3e4a63193d48bfd31abce8d8e0f10f42cc84a6b747fd768 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-26T02:11:49.260 INFO:journalctl@ceph.mds.l.smithi071.stdout:Apr 26 02:11:48 smithi071 bash[70815]: ceph-5d371fa8-036e-11ef-bc93-c7b262605968-mds-l 2024-04-26T02:11:49.411 DEBUG:teuthology.orchestra.run.smithi071:> sudo pkill -f 'journalctl -f -n 0 -u ceph-5d371fa8-036e-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T02:11:49.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T02:11:49.441 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T02:11:49.441 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:11:49.771 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T02:11:55.149 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:11:55.347 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T02:11:57.820 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T02:11:58.008 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T02:12:00.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:12:00.442 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:12:00.474 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T02:12:00.499 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T02:12:00.501 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi037/crash 2024-04-26T02:12:00.502 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:12:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:12:00.598 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:12:00.604 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi045/crash 2024-04-26T02:12:00.605 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:12:00.643 INFO:teuthology.orchestra.run.smithi045.stderr:tar: /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:12:00.644 INFO:teuthology.orchestra.run.smithi045.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:12:00.646 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi071/crash 2024-04-26T02:12:00.646 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash -- . 2024-04-26T02:12:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:tar: /var/lib/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T02:12:00.677 INFO:teuthology.orchestra.run.smithi071.stderr:tar: Error is not recoverable: exiting now 2024-04-26T02:12:00.678 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T02:12:00.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T02:12:00.813 INFO:tasks.cephadm:Compressing logs... 2024-04-26T02:12:00.813 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:12:00.818 DEBUG:teuthology.orchestra.run.smithi045:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:12:00.821 DEBUG:teuthology.orchestra.run.smithi071:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T02:12:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:12:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:: No such file or directory 2024-04-26T02:12:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:12:00.854 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr: -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr: 87.8% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.z.log: 90.4% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:12:00.855 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T02:12:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.4% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:12:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T02:12:00.856 INFO:teuthology.orchestra.run.smithi071.stderr: 94.4% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.f.log.gz/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.11.log: 2024-04-26T02:12:00.856 INFO:teuthology.orchestra.run.smithi071.stderr: 94.2% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T02:12:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:12:00.856 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T02:12:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T02:12:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:: No such file or directory 2024-04-26T02:12:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:12:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:12:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T02:12:00.862 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.7.log: /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.h.log: 90.4% 87.8% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr: -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.5% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:12:00.863 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:12:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.e.log: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T02:12:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T02:12:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T02:12:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T02:12:00.864 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.b.log: 92.0% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T02:12:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log: 92.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T02:12:00.865 INFO:teuthology.orchestra.run.smithi071.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:12:00.869 INFO:teuthology.orchestra.run.smithi045.stderr: 91.0% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:12:00.870 INFO:teuthology.orchestra.run.smithi071.stderr: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T02:12:00.876 INFO:teuthology.orchestra.run.smithi045.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T02:12:00.876 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose: No such file or directory -- 2024-04-26T02:12:00.876 INFO:teuthology.orchestra.run.smithi037.stderr: /var/log/ceph/cephadm.log 2024-04-26T02:12:00.877 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log 2024-04-26T02:12:00.878 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T02:12:00.879 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T02:12:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T02:12:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.g.log: 87.7% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T02:12:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T02:12:00.883 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T02:12:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T02:12:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T02:12:00.886 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T02:12:00.886 INFO:teuthology.orchestra.run.smithi037.stderr: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T02:12:00.886 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T02:12:00.888 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T02:12:00.891 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T02:12:00.892 INFO:teuthology.orchestra.run.smithi037.stderr: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T02:12:00.892 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.a.log: 93.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T02:12:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T02:12:00.895 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T02:12:00.897 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T02:12:00.902 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T02:12:00.902 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T02:12:00.906 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.2% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T02:12:00.912 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T02:12:00.952 INFO:teuthology.orchestra.run.smithi045.stderr: 93.6% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T02:12:01.145 INFO:teuthology.orchestra.run.smithi071.stderr: 92.9% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T02:12:01.318 INFO:teuthology.orchestra.run.smithi037.stderr: 90.0% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T02:12:01.342 INFO:teuthology.orchestra.run.smithi071.stderr: 91.8% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T02:12:01.404 INFO:teuthology.orchestra.run.smithi045.stderr: 91.9% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T02:12:01.686 INFO:teuthology.orchestra.run.smithi045.stderr: 93.2% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T02:12:01.837 INFO:teuthology.orchestra.run.smithi071.stderr: 93.3% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T02:12:01.874 INFO:teuthology.orchestra.run.smithi071.stderr: 93.0% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T02:12:01.925 INFO:teuthology.orchestra.run.smithi037.stderr: 93.0% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T02:12:02.007 INFO:teuthology.orchestra.run.smithi071.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T02:12:02.034 INFO:teuthology.orchestra.run.smithi037.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T02:12:02.051 INFO:teuthology.orchestra.run.smithi045.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T02:12:02.065 INFO:teuthology.orchestra.run.smithi045.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T02:12:02.147 INFO:teuthology.orchestra.run.smithi071.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T02:12:02.150 INFO:teuthology.orchestra.run.smithi071.stderr: 2024-04-26T02:12:02.150 INFO:teuthology.orchestra.run.smithi071.stderr:real 0m1.315s 2024-04-26T02:12:02.150 INFO:teuthology.orchestra.run.smithi071.stderr:user 0m5.117s 2024-04-26T02:12:02.150 INFO:teuthology.orchestra.run.smithi071.stderr:sys 0m0.212s 2024-04-26T02:12:02.172 INFO:teuthology.orchestra.run.smithi045.stderr: 93.2% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T02:12:02.174 INFO:teuthology.orchestra.run.smithi045.stderr: 2024-04-26T02:12:02.174 INFO:teuthology.orchestra.run.smithi045.stderr:real 0m1.340s 2024-04-26T02:12:02.175 INFO:teuthology.orchestra.run.smithi045.stderr:user 0m5.138s 2024-04-26T02:12:02.175 INFO:teuthology.orchestra.run.smithi045.stderr:sys 0m0.170s 2024-04-26T02:12:02.222 INFO:teuthology.orchestra.run.smithi037.stderr: 90.4% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T02:12:02.288 INFO:teuthology.orchestra.run.smithi037.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T02:12:02.551 INFO:teuthology.orchestra.run.smithi037.stderr: 93.1% -- replaced with /var/log/ceph/5d371fa8-036e-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T02:12:02.555 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-04-26T02:12:02.555 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m1.704s 2024-04-26T02:12:02.556 INFO:teuthology.orchestra.run.smithi037.stderr:user 0m6.573s 2024-04-26T02:12:02.556 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m0.578s 2024-04-26T02:12:02.557 INFO:tasks.cephadm:Archiving logs... 2024-04-26T02:12:02.557 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi037/log 2024-04-26T02:12:02.558 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:12:03.542 DEBUG:teuthology.misc:Transferring archived files from smithi045:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi045/log 2024-04-26T02:12:03.543 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:12:04.303 DEBUG:teuthology.misc:Transferring archived files from smithi071:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi071/log 2024-04-26T02:12:04.304 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T02:12:05.088 INFO:tasks.cephadm:Removing cluster... 2024-04-26T02:12:05.088 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --force 2024-04-26T02:12:05.423 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T02:12:05.995 DEBUG:teuthology.orchestra.run.smithi045:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --force 2024-04-26T02:12:06.198 INFO:teuthology.orchestra.run.smithi045.stdout:Deleting cluster with fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T02:12:06.469 DEBUG:teuthology.orchestra.run.smithi071:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 5d371fa8-036e-11ef-bc93-c7b262605968 --force 2024-04-26T02:12:06.658 INFO:teuthology.orchestra.run.smithi071.stdout:Deleting cluster with fsid: 5d371fa8-036e-11ef-bc93-c7b262605968 2024-04-26T02:12:06.920 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T02:12:06.920 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:12:06.983 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:12:06.999 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T02:12:07.013 INFO:tasks.cephadm:Teardown complete 2024-04-26T02:12:07.014 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T02:12:07.025 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T02:12:07.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:12:07.033 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:12:07.042 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T02:12:07.188 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T02:12:07.200 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T02:12:07.200 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:12:07.208 DEBUG:teuthology.orchestra.run.smithi045:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:12:07.210 DEBUG:teuthology.orchestra.run.smithi071:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T02:12:07.222 INFO:teuthology.orchestra.run.smithi045.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:12:07.224 INFO:teuthology.orchestra.run.smithi071.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:12:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:12:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:=============================================================================== 2024-04-26T02:12:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 +1728us[+1794us] +/- 52ms 2024-04-26T02:12:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -2000us[-2000us] +/- 73ms 2024-04-26T02:12:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 -588us[ -588us] +/- 82ms 2024-04-26T02:12:07.227 INFO:teuthology.orchestra.run.smithi045.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:12:07.229 INFO:teuthology.orchestra.run.smithi071.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:12:07.229 INFO:teuthology.orchestra.run.smithi071.stdout:=============================================================================== 2024-04-26T02:12:07.229 INFO:teuthology.orchestra.run.smithi071.stdout:^* hv01.front.sepia.ceph.com 3 6 377 44 +1751us[+1755us] +/- 52ms 2024-04-26T02:12:07.229 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 -1975us[-1972us] +/- 73ms 2024-04-26T02:12:07.229 INFO:teuthology.orchestra.run.smithi071.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -564us[ -560us] +/- 82ms 2024-04-26T02:12:07.229 INFO:teuthology.orchestra.run.smithi071.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:12:07.257 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-04-26T02:12:07.267 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T02:12:07.268 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-04-26T02:12:07.268 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 43 +1720us[+1777us] +/- 52ms 2024-04-26T02:12:07.268 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 44 -1989us[-1933us] +/- 73ms 2024-04-26T02:12:07.268 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 43 -587us[ -587us] +/- 82ms 2024-04-26T02:12:07.268 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T02:12:07.269 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T02:12:07.280 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T02:12:07.280 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T02:12:07.306 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:12:07.522 DEBUG:teuthology.orchestra.run.smithi045:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:12:07.592 DEBUG:teuthology.orchestra.run.smithi071:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T02:12:07.658 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:12:07.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:12:07.753 DEBUG:teuthology.orchestra.run.smithi045:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:12:07.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:12:07.783 DEBUG:teuthology.orchestra.run.smithi071:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T02:12:07.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:12:07.810 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T02:12:07.821 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T02:12:07.858 INFO:teuthology.task.internal:Duration was 2178.160694 seconds 2024-04-26T02:12:07.859 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T02:12:07.876 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T02:12:07.877 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:12:07.885 DEBUG:teuthology.orchestra.run.smithi045:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:12:07.887 DEBUG:teuthology.orchestra.run.smithi071:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T02:12:07.924 INFO:teuthology.orchestra.run.smithi071.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:12:07.925 INFO:teuthology.orchestra.run.smithi045.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:12:08.035 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T02:12:08.551 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T02:12:08.552 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-04-26T02:12:08.552 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:12:08.619 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi045.front.sepia.ceph.com 2024-04-26T02:12:08.620 DEBUG:teuthology.orchestra.run.smithi045:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:12:08.648 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi071.front.sepia.ceph.com 2024-04-26T02:12:08.649 DEBUG:teuthology.orchestra.run.smithi071:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T02:12:08.675 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T02:12:08.676 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:12:08.684 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:12:08.691 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T02:12:08.772 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T02:12:08.772 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:12:08.778 DEBUG:teuthology.orchestra.run.smithi045:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:12:08.780 DEBUG:teuthology.orchestra.run.smithi071:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T02:12:09.484 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T02:12:09.497 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T02:12:09.498 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:12:09.589 DEBUG:teuthology.orchestra.run.smithi045:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:12:09.617 DEBUG:teuthology.orchestra.run.smithi071:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T02:12:09.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T02:12:09.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:12:09.665 DEBUG:teuthology.orchestra.run.smithi045:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:12:09.668 DEBUG:teuthology.orchestra.run.smithi071:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:12:09.692 INFO:teuthology.orchestra.run.smithi045.stdout:kernel.core_pattern = core 2024-04-26T02:12:09.713 INFO:teuthology.orchestra.run.smithi071.stdout:kernel.core_pattern = core 2024-04-26T02:12:09.763 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-04-26T02:12:09.809 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:12:09.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:12:09.861 DEBUG:teuthology.orchestra.run.smithi045:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:12:09.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:12:09.878 DEBUG:teuthology.orchestra.run.smithi071:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T02:12:09.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T02:12:09.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T02:12:09.906 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T02:12:09.907 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi037 2024-04-26T02:12:09.907 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:12:10.020 DEBUG:teuthology.misc:Transferring archived files from smithi045:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi045 2024-04-26T02:12:10.020 DEBUG:teuthology.orchestra.run.smithi045:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:12:10.060 DEBUG:teuthology.misc:Transferring archived files from smithi071:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_01:02:28-fs-wip-lusov-quiescer-distro-default-smithi/7674259/remote/smithi071 2024-04-26T02:12:10.060 DEBUG:teuthology.orchestra.run.smithi071:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T02:12:10.100 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T02:12:10.100 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:12:10.108 DEBUG:teuthology.orchestra.run.smithi045:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:12:10.111 DEBUG:teuthology.orchestra.run.smithi071:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T02:12:10.162 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T02:12:10.178 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T02:12:10.178 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T02:12:10.210 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T02:12:10.210 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:12:10.218 DEBUG:teuthology.orchestra.run.smithi045:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:12:10.220 DEBUG:teuthology.orchestra.run.smithi071:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T02:12:10.236 INFO:teuthology.orchestra.run.smithi045.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:12 /home/ubuntu/cephtest 2024-04-26T02:12:10.237 INFO:teuthology.orchestra.run.smithi071.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:12 /home/ubuntu/cephtest 2024-04-26T02:12:10.271 INFO:teuthology.orchestra.run.smithi037.stdout: 393611 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 02:12 /home/ubuntu/cephtest 2024-04-26T02:12:10.274 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T02:12:10.286 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T02:12:10.353 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/iozone}} duration: 2178.160694360733 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-26T02:12:10.354 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T02:12:10.431 INFO:teuthology.run:pass